Patent Grant
7844111
The present invention relates to execution of predetermined correction regarding the image quality of color images captured by a digital camera, a scanner and the like. More particularly, the present invention relates to an image correction method to be carried out at the time of processing, editing and the like of color images, to a readable recording medium with a recorded processing program using the image correction method, and to an image processing apparatus to which the image correction method is applied.
In conventional computer processing, color images have been processed and edited by using predetermined program software. A program for correcting the image quality of color images has been known as a program for supporting this editing operation, and has been built in a special-purpose image processing apparatus or provided on the market in the form of ROM, for example.
Unlike the image correction of manuscript images captured by a color copier or the like, the image correction of color images in the processing and editing operations using a computer is required to reflect the intention of the editor. Therefore, in the processing procedure in accordance with a conventional image correction program, the necessity/nonnecessity of correction processing is judged by the editor. When image correction is instructed by the editor, the image correction program is executed, and predetermined correction processing is carried out.
In accordance with this flowchart, the editor displays plural images photographed by a digital camera for example on a display device, such as a CRT, and determines images required to be corrected. The editor inputs a command for instructing image correction to run an image correction program. In accordance with this image correction program, when plural files to be subjected to image correction and the output destinations of the files after correction are input by the editor interactively or in a form required for information input (#101 and #102), first, a processing file count value n is set at “1” (#103). Next, the image data of the file designated first is read (#104). After predetermined image correction processing (contour correction, contrast correction and the like, for example) is carried out (#105), the file including the image data is output to the designated output destination (a memory for storing the data, a printer and the like) (#106).
Next, the count value n is incremented by one (#107), and whether the count value n is larger than the designated number N of files is judged (#108). If the count value n is not larger than the number N (NO at #108), the procedure returns to #104, and the same predetermined image processing is carried out for the next image file (#104 to #108). If the count value n is larger than the designated total number N of files (YES at #108), it is judged that the image correction processing has been completed for the files, and the processing ends.
In the conventional image correction program, the editor judges the necessity/nonnecessity of image correction, and on the basis of the result of the judgment, the predetermined image correction processing is carried out for a color image required to be corrected. Therefore, in the case when plural photographed color images are corrected for example, the editor must judge the necessity/nonnecessity of image correction for each color image. This becomes burdensome to the editor. In addition, since the designated plural color images are corrected all together, in the case when images not required to be corrected are included, image quality will be deteriorated by the correction, and image correction must be carried out again. This may lower processing efficiency.
In order to solve the above-mentioned problems, the present invention is intended to provide an image processing method capable of carrying out image correction efficiently, a readable recording medium with a recorded program for the processing method and an image processing apparatus to which the processing method is applied.
The present invention relates to an image processing method for carrying out predetermined correction processing with respect to one or more items regarding the quality of color images. The image processing method judging the necessity of correction with respect to each of the above-mentioned items and a correction processing for carrying out the predetermined correction processing with respect to the item judged as required to be corrected during judgment.
With the above-mentioned configuration, in the case when the predetermined image correction processing is carried out with respect to one or more items regarding the image quality of color images, unlike the case when correction is carried out in comparison with predetermined values having been determined simply, the necessity/nonnecessity of correction is judged comprehensively with respect to each item on the basis of the whole condition of each image. Correction processing is then carried out with respect to items required to be corrected.
In addition, the present invention includes, as correction judgment items, at least one of a sunset judgment used to judge whether a color image is a sunset scene or not, a color covering judgment used to judge whether the whole of a color image is covered with a specific color or not, a contrast judgment used to judge whether a color image has a normal contrast or not, and a sharpness judgment used to judge whether a color image has a normal sharpness or not.
With the above-mentioned configuration, at least one of the sunset judgment, color covering judgment, contrast judgment and sharpness judgment is carried out. When it is judged that correction is necessary, the predetermined correction processing with respect to the judgment item is carried out for a color image.
Furthermore, the present invention relates to a readable recording medium with a recorded program for carrying out the predetermined correction processing with respect to one or more items regarding the image quality of color images. The processing program comprising judging the necessity of correction with respect to each item described above and correction processing for carrying out the predetermined correction processing with respect to the item judged as required to be corrected at the above-mentioned judgment is stored in the readable recording medium.
Moreover, the present invention relates to an image processing apparatus for carrying out the predetermined correction processing with respect to one or more items regarding the image quality of color images. The image processing apparatus comprises a judgment device to judge the necessity of correction with respect to each item described above and a correction device to carry out the predetermined correction processing with respect to the item judged as required to be corrected during judgment.
Additionally, the image processing apparatus of the present invention comprises an image designation device to designate plural color images to be corrected, and a correction processing control device to judge the necessity of correction for the color images designated by the image designation device and for carrying out the predetermined correction processing on the basis of the result of the judgment.
These and other objects, advantages and features of the invention will become apparent from the following description thereof taken in conjunction with the accompanying drawings which illustrate specific embodiments of the invention.
In the following description, like parts are designated by like reference numbers throughout the several drawings.
Embodiments in accordance with the present invention shown in the accompanying drawings will be described below in detail.
The image processing apparatus shown in
The control unit 2 is provided with an image file input/output processing unit 201, a correction necessity/nonnecessity judgment unit 202 and an image correction unit 203 for carrying out image processing.
The image file input/output processing unit 201 is used to read color images (electric images) of an image file having been input from the input device 8 from a recording medium (the RAM 4, the external storage device 10, an internal storage device if provided, or the like, for example), and to output corrected image data to a predetermined output destination (a recording medium, a printer, other peripheral devices or the like) having been input from the input device 8.
The correction necessity/nonnecessity judgment unit 202 is used to judge whether preset image correction is required or not for color images read from a recording medium. The correction necessity/nonnecessity judgment unit 202 has at least a sunset judgment, a color covering judgment, a contrast judgment and a sharpness judgment as judgment items. The sunset judgment is used to judge whether a color image is a sunset scene or not, and the color covering judgment is used to judge whether the whole of a color image is covered with a specific color (a greenish color when a photograph is taken by using a fluorescent lamp as a light source, for example) or not. In addition, the contrast judgment is used to judge whether a color image has a normal contrast or not, and the sharpness judgment is used to judge whether a color image has a normal sharpness or not.
The image correction unit 203 is used to carry out image correction for items corresponding to a color image judged as “correction necessary” on the basis of the judgment result of the correction necessity/nonnecessity judgment unit 202. If an image different from a sunset scene is judged that the image is covered with red for example, the red component on the whole of the color image is weakened to correct the color covering.
The ROM 3 is a memory wherein the program for the above-mentioned image correction processing has been stored. The RAM 4 is used to temporarily store various data calculated by the image correction processing. Furthermore, the image memory 5 is used to store color image data read from a recording medium to carry out image correction judgment and image correction. The image memory 5 has an image data storage capacity for at least three images. Image data constituting color images is separated into the color components R, G and B, and stored.
The display device 9 is used to display various matters, such as an operation menu, processing conditions and processing results (including monitor display of corrected color images), and is formed of an electronic display, such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display). Image correction items are represented by icons on the operation menu of the display device 9, whereby the operator can carry out image correction processing as described below by selecting the icons.
The external storage device 10 functions as a storage device for image data to be corrected or a storage device for corrected image data, and reads and writes image data on external recording media such as floppy disks, optical disks, magnetic disks and magneto-optic disks.
Next, the image correction processing of the image processing apparatus will be described below referring to the flowcharts of
When plural image files for image correction and an output destination after image correction of each image file are input by the operator (#1 and #3), a processing file count value n is first set at “1” (#5). Next, the image data of the first image file is read from a designated recording medium to the image memory 5 (#7). The image data is subjected to the judgment of the necessity/nonnecessity of preset image correction items in accordance with the correction necessity/nonnecessity judgment processing procedure shown in
When image correction is judged as necessary on the basis of the image correction necessity/nonnecessity judgment (YES at #11), predetermined image correction processing is carried out for items judged as “correction necessary,” and the corrected image data is then output to a designated output destination (the external storage device 10, a printer (not shown) or the like, for example) (#13 and #15). Furthermore, when image correction is judged as unnecessary (NO at #11), image data is output as it is to the designated output destination without image correction processing (#15).
Next, the count value n is incremented by only “1” (#17), and whether the count value n is larger than the total number N of the designated image files or not is judged (#19). If the count value n is not larger (NO at #19), the procedure returns to step #7, and the image correction necessity/nonnecessity judgment, the image correction processing and the output processing of image data are carried out for the next image file name in the same way (#7 to #15). If the count value n is larger than the total number N of the files (YES at #19), it is judged that image correction processing for all the image files has been completed, and the processing ends.
Next, the image correction necessity/nonnecessity judgment will be described in accordance with the flowchart shown in
In the image correction necessity/nonnecessity judgment, by using the image data stored in the image memory 5, a judgment as to whether the color image thereof is an image of a sunset scene or not (#21 and #23), a judgment as to whether the color image is color-covered or not (#25 and #27), a judgment as to whether the color image is normal in contrast or not (#29 and #31), and a judgment as to whether the color image is normal in sharpness or not (#33 and #35) are carried out in this order. This order of judgment items is determined to raise judgment efficiency and judgment accuracy.
In other words, the judgment as to whether the color image is a sunset scene or not (hereinafter referred to as a sunset judgment) is carried out on the basis of the histogram distribution of image data of part of color components in the range of red to yellow as described below. The judgment as to whether the color image is color-covered or not (hereinafter referred to as a color covering judgment) is carried out on the basis of the histogram distribution of image data of the whole of color components in the range of red to blue as described below. Since the processing load for the color covering judgment is higher than that for the sunset judgment, the sunset judgment is carried out earlier than the color covering judgment to attain efficient judgment.
Furthermore, the judgment as to whether the color image is normal in contrast or not (hereinafter referred to as a contrast judgment) is carried out after the sunset judgment and the color covering judgment in view of judgment efficiency, since the contrast judgment can use the histograms of the image data created at the time of the sunset judgment and the color covering judgment. A photographed image has sharp edges just as a photographed scene taken in cloudy weather although its contrast is low. In order to be able to judge the contents of such a photographed scene, the judgment as to whether the color image is normal in sharpness or not (hereinafter referred to as a sharpness judgment) is carried out after the contrast judgment in consideration of judgment accuracy.
At the time of the sunset judgment, an index value for the sunset judgment is calculated by using the histogram of hue data (#21) as described below, and this index value is compared with a predetermined threshold value to judge whether the color image is a sunset scene or not (#23). If the color image is a sunset scene (YES at #23), a judgment result of “correction unnecessary” is output (#39). The procedure returns, and if the color image is not a sunset scene (NO at #23), the color image is further subjected to the color covering judgment.
At the time of the color covering judgment, an index value for the color covering judgment is calculated by using the histogram of the hue data (#25) as described below, and this index value is compared with a predetermined threshold value to judge whether the color image has undergone the covering of a specific color or not (#27). If the color image is not a sunset scene and is color-covered (YES at #27), a judgment result requiring correction for color covering is output (#37). The procedure returns, and if the color image is not a sunset scene and is not color-covered (NO at #27), the contrast judgment is further carried out (#25).
At the time of the contrast judgment, an index value for the contrast judgment is calculated by using the histograms of the color components R, G and B (#29) as described below, and this index value is compared with a predetermined threshold value to judge whether the contrast of the color image is normal or not (#31). If the color image is not a sunset scene, is not color-covered and is normal in contrast (NO at #31), the color image is judged that it is not a sunset scene and is normal in image quality, and a judgment result of “correction unnecessary” is output (#39). The procedure returns, and if the color image is not a sunset scene and is not color-covered but is abnormal in contrast (YES at #31), the sharpness judgment is further carried out (#35).
At the time of the sharpness judgment, data in the edge region in the color image is extracted, an index value for the sharpness judgment is calculated by using this data (#33), and this index value is compared with a preset threshold value to judge whether the sharpness of the color image is normal or not (#35).
If the color image is not a sunset scene, is not color-covered and is normal in both contrast and sharpness (NO at #35), the color image is judged that it is not a sunset scene and is normal in image quality, and a judgment result of “correction unnecessary” is output (#39). The procedure returns, and if the color image is not a sunset scene and is not color-covered but is abnormal in contrast and sharpness (YES at #35), a judgment result requiring correction for intensifying image edges is output (#37). The procedure then returns.
Next, the sunset judgment procedure will be described referring to the flowchart of
In the case of the photographed image of a scene wherein a white wall is irradiated with evening sunlight for example, the white wall is orange-colored, and the shade of the color changes spontaneously in the range of red to yellow. Therefore, the sunset judgment is carried out by judging the distribution condition of the color components in the range of red to yellow in the image. More concretely, the sunset judgment is carried out as described below. Regarding the image data (pixel data) belonging in the range from red to yellow, the product P (=H·S) of the hue data H and the saturation data S thereof, and the product Q (=H·L) of the hue data H and the lightness data L are calculated, P and Q are represented by histograms, and the degrees of their dispersion are examined.
In other words, the image data (pixel data) of the color components R, G and B stored in the image memory 5 is first converted into the three attributes of color, i. e., H, L and S, in accordance with the conversion equation shown at the following steps 0 to 3 (#41).
Step 0 (Preparation)
Note that at least one of r, g and b is 0, and that at least one of them is 1.
Step 1 (Calculation of Lightness L)
Step 2 (Calculation of Saturation S)
Step 3 (Calculation of Hue H)
Next, after the level of the hue data H is converted so that the maximum value 360 becomes “255”, the histogram of the hue data H is created at a class pitch of “1” for example (#43). The hue data H is normalized herein by “255” because of the following reasons. Since data processing is carried out by using 8-bit data, the processing can be made easy by this normalization. Furthermore, the judgment accuracy is not much affected by this normalization.
Next, the hue data H included in the range of class 64 to class 125 (in the range of red to yellow) is extracted. The product P (=H·S) of each of the hue data H and the saturation data S corresponding thereto and the product Q (=H·L) of each of the hue data H and the lightness data L corresponding thereto are calculated (#45).
Next, after the levels of the calculated values P and Q are converted so that the maximum value becomes “255” (#47), the histograms of the calculated values P and Q are created at a class pitch of “1” for example (#49), and the variance values Sp and Sq of each histogram are calculated (#51). The variance value S can be calculated by using the following equation (1) for a histogram with frequency y corresponding to class x as shown in
The variance values Sp and Sq are compared with predetermined threshold values Kp and Kq (Kp=850, Kq=1200) corresponding thereto, respectively (#53 and #55). If Sp≧Kp and Sq≧Kq (YES at #53 and #55), it is assumed that the gradation at the portions in the range of red to yellow in a color image is wide, and that the red portion is colored by evening sunlight, whereby the color image is judged as a sunset image (#57). If Sp<Kp or Sq<Kq (NO at #53 or NO at #55), it is assumed that the gradation at the portions in the range of red to yellow in the color image is narrow, and that the red portion is not colored by evening sunlight, whereby the color image is not judged as a sunset image (#59).
Next, the color covering judgment procedure will be described below referring to the flowchart of
Color covering is a condition wherein the whole of a color image is covered with a specific color and the color balance of the color image is lost such that a photograph taken without flashlight under a fluorescent lamp for example becomes greenish on the whole. Therefore, the color covering judgment is carried out by judging the distribution condition of all the color components in the image. However, since the contents of the image may sometimes have one specific color, the color covering judgment is carried out in this case in consideration of not misjudging that the image is color-covered.
First, the image data (pixel data) of the color components R, G and B stored in the image memory 5 is converted into the image data of the three color attributes H, L and S by the conversion equation shown as the above-mentioned equation 1 (#61). Next, image data having the lightness data L in the range of about “15” to “85” is extracted (#63). Since the shade of the color cannot be judged accurately at the dark portions and bright portions in the image, this extraction is carried out so that the image data in these portions is not judged.
Next, a histogram is created for the hue data H corresponding to the extracted lightness data L at a class pitch of “1” for example (#65). A judgment is carried out as to whether a predetermined quantity or more (85% or more for example) of the whole image data (pixel data) is intensively distributed in a predetermined range W1 (the range of class 20 for example) of a specific color region (#67). Since the presence or absence of color covering is checked in the whole color region in the case of the color covering judgment, the processing of normalizing the hue data H by “255” is not carried out in consideration of judgment accuracy.
If image data is intensively distributed in a specific color region as shown in
If 80% or more of image data (pixel data) is intensively distributed in the predetermined range W2 of a specific color region as shown in
Next, the contrast judgment procedure will be described below referring to the flowchart of
The contrast judgment is carried out by judging whether the image data of each of the color components R, G and B effectively has used a dynamic range or not. In other words, in the case when the histogram of the image data is created as shown in
First, a histogram at the level of the class pitch “1” for example is created for each of color components R, G and B (#81). Next, the ranges dr, dg and db of the frequency “0” class are calculated for the color components, respectively (#83), and the ratio dj/D (%) (j=r, g, b) of the class range dr, dg, db to the whole class range D (=255) is calculated (#85). And dj/D is compared with a predetermined threshold value Kj (j=r, g, b) (10% for example), respectively (#87, #89 and #91). If Dj/D≦Kj in all the color components of the image (YES at all the steps of #87 to #91), the contrast is judged as normal (#93). If Dj/D>Kj (NO at #87, #89 or #91), the contrast is judged as abnormal (#95).
Although the image data of the color components R, G and B is used for the contrast judgment shown in
Next, the sharpness judgment will be described.
The sharpness judgment is carried out by calculating sharpness by using the method disclosed in the U.S. Pat. No. 2,611,723 for example, and by comparing this sharpness with a predetermined threshold value. The details of the method of calculating the sharpness are omitted. The sharpness disclosed in the patent publication is obtained by extracting the image data (pixel data) of the edge portions of an image by using a linear differentiation filter, a quadratic differentiation filter, a high frequency filter or a band-pass filter, by calculating a high frequency component at a pixel position extracted by applying a high frequency filter to this image data, and by normalizing the intensity of the high frequency component of the whole edge region obtained by integrating the intensity of the high frequency component with respect to the area of the edge region.
If the sharpness is higher than the predetermined threshold value, the sharpness is judged as normal. If the sharpness is not higher than the predetermined threshold value, the sharpness is judged as abnormal.
As described above, in the color image correction processing, the necessity/nonnecessity of correction is judged with respect to the correction items regarding the preset image quality, and image correction is carried out only for the items required to be corrected. Therefore, correction is not carried out for the color images which are not required to be corrected, whereby processing efficiency can be enhanced. Furthermore, it is possible to prevent image deterioration due to image correction carried out for color images which are not required to be corrected.
In the above-mentioned embodiment, the sunset judgment, the color covering judgment, the contrast judgment and the sharpness judgment are taken as examples of the judgment items for the necessity/nonnecessity of correction in image quality. However, if the saturation of a color image is excessive for example, a judgment as to whether correction for relieving the excessive saturation is required or not may be included. The judgment as to whether the saturation is normal or not is carried out by converting the image data of the three primary colors R, G and B into the image data of the three attributes H, S and L for example, by selecting a predetermined hue, and by comparing the saturation data S corresponding to the hue with a predetermined threshold value Ks (50% of the maximum saturation of the hue for example). If the saturation data is not higher than the threshold value Ks for example, the saturation is judged as abnormal, and saturation correction processing is carried out. The saturation of the hue is corrected so that the saturation data S becomes 60% of the maximum saturation of the hue for example.
Furthermore, in the case when image data has much noise (the noise is not limited to signal noise, but includes improper image data caused owing to loss or deterioration of image data and required to be excluded from desired image data), a judgment as to whether correction for eliminating the noise is required or not may be included.
In addition, although photographed images are taken as examples in the explanation of the above-mentioned embodiment, the present invention can be applied to any images, such as images created by computer graphics, images captured by scanners and the like.
Furthermore, although the image processing apparatus provided with the processing program for the image processing method in accordance with the present invention is described in the above-mentioned embodiment, it is possible to configure an image processing apparatus 20 as shown in
As described above, in accordance with the present invention, when the image quality of a color image is corrected, the necessity/nonnecessity of correction is judged, and the predetermined image processing is carried out only when the correction is necessary. Therefore, the necessity/nonnecessity of image correction is not required to be judged by the operator, whereby the processing efficiency of image correction is improved by that amount. In addition, in the case when correction is carried out for plural color images all together, the present invention is free from a problem of causing image deterioration due to unnecessary image correction which may otherwise be carried out for color images not required to be corrected.
Furthermore, since at least one of the sunset judgment, the color covering judgment, the contrast judgment and the sharpness judgment is included as a correction judgment item, the necessity/nonnecessity judgment for proper image correction can be carried out for photographed color images.
Moreover, when the image quality of a color image is corrected, the necessity/nonnecessity of correction is judged, and only when the correction is necessary, the processing program for carrying out the predetermined image processing is recorded on a readable recording medium. By applying this readable recording medium to a computer system, an image processing apparatus can be configured simply.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-102756 | Apr 1998 | JP | national |
This application is a divisional application of Ser. No. 11/727,560, filed Mar. 27, 2007, which is a divisional application of Ser. No. 09/291,358, filed Apr. 14, 1999, now U.S. Pat. No. 7,215,812, issued May 8, 2007, which claims priority from Japanese patent application No. 10-102756, filed Apr. 14, 1998, the contents of which are hereby incorporated by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4364084 | Akimoto et al. | Dec 1982 | A |
| 4651199 | Alkofer | Mar 1987 | A |
| 4989079 | Ito | Jan 1991 | A |
| 5109275 | Naka et al. | Apr 1992 | A |
| 5194946 | Morikawa et al. | Mar 1993 | A |
| 5202935 | Kanamori et al. | Apr 1993 | A |
| 5317420 | Kuwahara | May 1994 | A |
| 5347374 | Fuss et al. | Sep 1994 | A |
| 5418895 | Lee | May 1995 | A |
| 5448502 | Kindo et al. | Sep 1995 | A |
| 5450502 | Eschbach et al. | Sep 1995 | A |
| 5461440 | Toyoda et al. | Oct 1995 | A |
| 5508823 | Kiyohara et al. | Apr 1996 | A |
| 5543936 | Ulichney | Aug 1996 | A |
| 5576811 | Kobayashi et al. | Nov 1996 | A |
| 5579131 | Kusumoto et al. | Nov 1996 | A |
| 5667944 | Reem et al. | Sep 1997 | A |
| 5742410 | Suzuki | Apr 1998 | A |
| 5751343 | Hibino et al. | May 1998 | A |
| 5799105 | Tao | Aug 1998 | A |
| 5946006 | Tajika et al. | Aug 1999 | A |
| 5974100 | Maeda et al. | Oct 1999 | A |
| 5999230 | Tanabe | Dec 1999 | A |
| 6014469 | Eschbach | Jan 2000 | A |
| 6055340 | Nagao | Apr 2000 | A |
| 6151410 | Kuwata et al. | Nov 2000 | A |
| 6181321 | Zhao et al. | Jan 2001 | B1 |
| 6201893 | Shiraiwa et al. | Mar 2001 | B1 |
| 6256112 | Kawano | Jul 2001 | B1 |
| 6272259 | Mizoguchi | Aug 2001 | B1 |
| 6388768 | Hada et al. | May 2002 | B2 |
| 6393148 | Bhaskar | May 2002 | B1 |
| Number | Date | Country |
|---|---|---|
| 0 357 385 | Mar 1990 | EP |
| 2105677 | Apr 1990 | JP |
| 5101181 | Apr 1993 | JP |
| 5191661 | Jul 1993 | JP |
| 6-68253 | Mar 1994 | JP |
| 9172553 | Jun 1997 | JP |
| 9-322005 | Dec 1997 | JP |
| 09322002 | Dec 1997 | JP |
| 10095135 | Apr 1998 | JP |
| Number | Date | Country | |
|---|---|---|---|
| 20090161186 A1 | Jun 2009 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 11727560 | Mar 2007 | US |
| Child | 12389126 | US | |
| Parent | 09291358 | Apr 1999 | US |
| Child | 11727560 | US |
