Image processing method and system

Abstract

An original image signal made up of a plurality of image signal components each representing a pixel of an original image is processed to suppress the granularity of the original image and enhance the sharpness of the original image, and a final processed image signal is obtained. Processing for suppressing granularity and processing for enhancing sharpness are carried out separately on the original image signal to obtain a granularity-suppressed image signal and a sharpness-enhanced image signal. At least one intermediate processed image signal is obtained by operation based on at least the granularity-suppressed image signal and the sharpness-enhanced image signal, and, as the image signal component for each pixel of the final processed image signal, the image signal component for the corresponding pixel of an image signal selected from the intermediate processed image signals and the original image signal is taken according to an indicator value calculated on the basis of the granularity-suppressed image signal, the sharpness-enhanced image signal and the original image signal.

Description

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to an image processing method and an image processing system, and more particularly to an image processing method of and an image processing system for processing an image to enhance sharpness while suppressing granularity (high frequency noise).

2. Description of the Related Art

There has been known a system in which an image signal is obtained by reading an image on a photographic film or a printed image (including a color image) by an image sensor such as a CCD, the image signal is variously processed and an image is printed or displayed on a CRT or the like on the basis of the processed image signal. Such image processing includes, for instance, frequency processing which acts on predetermined frequency components of the image, and gradation processing which acts on the density of the image.

As the frequency processing, sharpness enhancement processing for suppressing blur of the edges of an image, granularity suppressing processing such as smoothening processing for suppressing granularity due to the granularity value of the photosensitive material and the like have been known.

As the sharpness enhancement processing, there have been known unsharp masking processing, high frequency band enhancement filter processing, processing for enhancing high frequency components of the image while suppressing intermediate frequency components (Japanese Unexamined Patent Publication No. 9(1997)-22460), and the like. As the granularity suppressing processing, there have been known, for instance, a median filter processing, hysteresis smoothing processing, noise removing processing by repetition, and granularity suppressing processing using morphology operation.

When the sharpness enhancement processing is carried out, granularity of the image is also enhanced though the sharpness of the image is improved, whereas when the granularity suppressing processing is carried out, the sharpness of the image is deteriorated though the granularity of the image is suppressed.

Accordingly, there has been a demand for image processing which can enhance the sharpness of the image while suppressing the granularity of the image, and there have been proposed various processings which can enhance the sharpness of the image and at the same time can suppress the granularity of the image as disclosed, for instance, in U.S. Pat. No. 4,812,903 and Japanese Unexamined Patent Publication Nos. 63(1988)-26783 and 9(1997)-22460. However, it should be said that the conventional techniques fail in effectively realizing enhancement of the sharpness together with suppression of the granularity.

For example, the technique disclosed in Japanese Unexamined Patent Publication No. 9(1997)-22460 tries to realize enhancement of the sharpness together with suppression of the granularity by enhancing high frequency components which affect the sharpness of the image and at the same time suppressing intermediate frequency components which affect the granularity of the image. This technique utilizes the fact that the granular portion is low in the degree of correlation with color and the image signal (edge) portion is high in the degree of correlation with color. In this technique, the image is developed into frequency ranges and the strength of intermediate to high frequency components is controlled so that the portion which is low in the degree of correlation with color is less enhanced, thereby suppressing the granularity. However in this processing, since continuity between adjacent pixels in a real image is not taken into account, abrupt change in the degree of control occurs in some part of the image, where the granularity cannot be sufficiently suppressed. Accordingly, this processing fails in effectively suppressing the granularity though it can effectively enhance the sharpness.

SUMMARY OF THE INVENTION

In view of the foregoing observations and description, the primary object of the present invention is to provide an image processing method and system which can effectively realize enhancement of the sharpness together with suppression of the granularity.

The image processing method and system of the present invention are characterized in that processing for suppressing granularity and processing for enhancing sharpness are separately carried out on an original image signal to obtain a granularity-suppressed image signal and a sharpness-enhanced image signal, at least one intermediate processed image signal is obtained by processing the image signal components of the granularity-suppressed image signal and the sharpness-enhanced image signal as well as the original image signal, as required, in one-to-one correspondence, and as the image signal component for each pixel of the final processed image signal, the image signal component for the corresponding pixel of an image signal selected from the original image signal and the intermediate processed image signal(s) is taken according to an indicator value defined by the granularity-suppressed image signal, the sharpness-enhanced image signal and the original image signal.

That is, in accordance with one aspect of the present invention, there is provided a method of processing an original image signal f in made up of a plurality of image signal components each representing a pixel of an original image (inclusive of a color image) to suppress the granularity of the original image and enhance the sharpness of the original image, and thereby obtaining a final processed image signal f out , the method comprising the steps of

carrying out processing for suppressing granularity and processing for enhancing sharpness separately on the original image signal f in to obtain a granularity-suppressed image signal f 1 and a sharpness-enhanced image signal f 2 ,

obtaining at least one intermediate processed image signal f 3 (f 4 , f 5 , . . . ) by operation based on at least the granularity-suppressed image signal and the sharpness-enhanced image signal, and

taking, as the image signal component for each pixel of the final processed image signal f out , the image signal component for the corresponding pixel of an image signal selected from the intermediate processed image signal(s) f 3 and the original image signal f in according to an indicator value k(f 1 , f 2 , f in ) calculated on the basis of the granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in .

In this specification, f in , f 1 , f 2 , f out and the like are sometimes used to denote image signal components of the corresponding image signals, which will be apparent to those skilled in the art. As the indicator value k(f 1 , f 2 , f in ), for instance, (f 1 −f in )·(f 2 −f in ) may be used. In the case where one intermediate processed image signal f 3 is obtained, the image signal component of the original image signal f in is taken as the image signal component of the final processed image signal f out when the indicator value (f 1 −f in )·(f 2 −f in ) is smaller than 0, and the image signal component of the intermediate processed image signal f 3 is taken as the image signal component of the final processed image signal f out when the indicator value (f 1 −f in )·(f 2 −f in ) is not smaller than 0 as represented by the following formula (1).

In the case where a pair of intermediate processed image signals f 3 and f 4 are obtained, the image signal component of the first intermediate processed image signal f 3 is taken as the image signal component of the final processed image signal f out when the indicator value (f 1 −f in )·(f 2 −f in ) is smaller than 0, and the image signal component of the second intermediate processed image signal f 4 is taken as the image signal component of the final processed image signal f out when the indicator value (f 1 −f in )·(f 2 −f in ) is not smaller than 0 as represented by the following formula (4). $$ f out = { f in ⁢   ⁢ ( when ⁢   ⁢ ( f1 - f in ) · ( f2 - f in ) < 0 ) , f3 ⁢   ⁢ ( when ⁢   ⁢ ( f1 - f in ) · ( f2 - f in ) ≧ 0 ) ( 1 )

wherein f in represents the original image signal, f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal, and f 3 represents the intermediate processed image signal. $$ f out = { f3 ⁢   ⁢ ( when ⁢   ⁢ ( f1 - f in ) · ( f2 - f in ) < 0 ) , f4 ⁢   ⁢ ( when ⁢   ⁢ ( f1 - f in ) · ( f2 - f in ) ≧ 0 ) ( 4 )

wherein f in represents the original image signal, f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal, f 3 represents the first intermediate processed image signal, and f 4 represents the second intermediate processed image signal.

That is, formula (1) represents processing in which the image signal component of the intermediate processed image signal f 3 is taken as the image signal component of the final processed image signal f out when both the image signal components of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 are larger or smaller than the image signal component of the original image signal f in or when the image signal component of at least one of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 is equal to the image signal component of the original image signal f in , and the image signal component of the original image signal f in is taken as the image signal component of the final processed image signal f out when the image signal component of one of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 is smaller than the image signal component of the original image signal f in and the image signal component of the other of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 is larger than the image signal component of the original image signal f in . At portions such as an edge portion of an image where the density abruptly changes, the indicator value (f 1 −f in )·(f 2 −f in ) is generally not smaller than 0 ((f 1 −f in )·(f 2 −f in )≧0), and accordingly the intermediate processed image signal f 3 where the sharpness is higher than the original image signal f in is selected. On the other hand, at the other portions (e.g., flat density portions), the indicator value (f 1 −f in )·(f 2 −f in ) is generally smaller than 0 ((f 1 −f in )·(f 2 −f in )<0), and accordingly, the original image signal f in is selected since sufficient granularity suppressing effect cannot be obtained with the intermediate processed image signal f 3 which is a weighted average of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 . In this case, by selecting the original image signal f in , a granularity-suppressed final processed image signal f out can be obtained. Thus, the granularity of the image can be suppressed without deteriorating the sharpness of the edge portion and the like.

In the case where three intermediate processed image signals f 3 , f 4 and f 5 are obtained, the image signal component of one of the first to third intermediate processed image signal f 3 , f 4 and f 5 is taken as the image signal component of the final processed image signal f out according to whether the indicator value k(f 1 , f 2 , f in ) is positive or negative and the value of |f in −f 1 | as represented by the following formula (15).

f out =f 3 (when | f in− f 1 |≧ Th )

f out =f 4 (when | f in− f 1 |< Th and ( f 1 − f in)·( f 2 − f in)<0)

f out =f 5 (when | f in− f 1 |< Th and ( f 1 − f in)·( f 2 − f in)≧0)  (15)

wherein f in represents the original image signal, f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal, f 3 represents the first intermediate processed image signal, f 4 represents the second intermediate processed image signal, f 5 represents the third intermediate processed image signal, and Th represents a constant.

The operation for obtaining the intermediate processed image signal f 3 may be selected from a plurality of kinds of preset operations according to the relation between the intermediate processed image signal f 3 and the original image signal f in , the kind of the image and/or the position of the pixel, or may be input from the external for each processing. Specifically the operation is preferably that defined by the following formula (2) which is based on the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 , or that defined by the following formula (3) which is based on the granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in .

f 3 = w 1 · f 1 + w 2 · f 2   (2)

wherein w 1 +w 2 =1 (0<w 1 <1, 0<w 2 <1), f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal and f 3 represents the intermediate processed image signal.

f 3 ={| f 1 − f in |( f 1 − f in )+| f 2 − f in |( f 2 − f in )}/{| f 1 − f in |+|f 2 − f in |}+f in   (3)

wherein f in represents the original image signal, f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal and f 3 represents the intermediate processed image signal.

That is, in accordance with formula (2), the intermediate processed image signal f 3 is obtained by weighted addition of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 , and by weighted average of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 , the sharpness can be enhanced while suppressing the granularity. Also in this operation, the weighting factors w 1 and w 2 may be changed pixel to pixel according to the relation between the intermediate processed image signal f 3 and the original image signal f in , or the like, or may be common to all the pixels.

Further each of the weighting factors w 1 and w 2 is preferably 0.2 to 0.8, more preferably 0.3 to 0.7 and most preferably 0.4 to 0.6.

In accordance with formula (3), the image signal components of the sharpness-enhanced image signal f 2 are taken as the image signal components of the intermediate processed image signal f 3 for pixels where the image signal components of the granularity-suppressed image signal f 1 are equal to those of the original image signal f in , the image signal components of the granularity-suppressed image signal f 1 are taken as the image signal components of the intermediate processed image signal f 3 for pixels where the image signal components of the sharpness-enhanced image signal f 2 are equal to those of the original image signal f in , and values obtained by weighting the differences between the image signal components of the granularity-suppressed image signal f 1 and the original image signal f in and between the image signal components of the sharpness-enhanced image signal f 2 and the original image signal f 1 and adding the weighted differences are taken as the image signal components of the intermediate processed image signal f 3 for pixels where the image signal components of neither of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 are equal to those of the original image signal f in . With this operation, for the image signal components for pixels in the flat density portion (an area of the image where change of the density or brightness is less) where the granularity is more apt to be remarked than the sharpness, the image signal components of the granularity-suppressed image signal f 1 is more weighted and accordingly the granularity can be suppressed. To the contrast, for the image signal components for pixels in the edge portion (an area of the image where change of the density or brightness is sharp) where the sharpness is more apt to be remarked than the granularity, the image signal components of the sharpness-enhanced image signal f 2 is more weighted and accordingly the sharpness can be enhanced. This operation is a specific example of the operation where processing is selected pixel to pixel according to the relation with the original image signal f in .

Formula (4) represents processing in which the image signal component of the second intermediate processed image signal f 4 is taken as the image signal component of the final processed image signal f out when both the image signal components of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 are larger or smaller than the image signal component of the original image signal f in or when the image signal component of at least one of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 is equal to the image signal component of the original image signal f in (k≧0), and the image signal component of the first intermediate processed image signal f 3 is taken as the image signal component of the final processed image signal f out when the image signal component of one of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 is smaller than the image signal component of the original image signal f in and the other of the image signal components of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 is larger than the image signal component of the original image signal f in (k<0).

It is preferred that the degree of granularity suppression of the first intermediate processed image signal f 3 be higher than that of the second intermediate processed image signal f 4 , and the degree of sharpness enhancement of the second intermediate processed image signal f 4 be higher than that of the first intermediate processed image signal f 3 . At portions such as an edge portion of an image where the density abruptly changes, the indicator value k is generally not smaller than 0 (k≧0), and accordingly by selecting the second intermediate processed image signal f 4 which is higher than the first intermediate processed image signal f 3 in the degree of sharpness enhancement, the edge portion can be prevented from becoming unsharp. On the other hand, at the other portions (e.g., flat density portions), the indicator value k is generally smaller than 0 (k<0) and the granularity is apt to be remarked, and accordingly, by selecting the first intermediate processed image signal f 3 which is higher than the second intermediate processed image signal f 4 in the degree of granularity suppression, the granularity can be suppressed.

It is preferred that the first and second intermediate processed image signals f 3 and f 4 be obtained by weighting the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 with weighting factors w 1 a, w 1 b, w 2 a and w 2 b and adding the weighted image signals f 1 and f 2 as represented by the following formulae (5) and (6).

f 3 = w 1 a·f 1 + w 1 b·f 2   (5)

f 4 = w 2 a·f 1 + w 2 b·f 2   (6)

wherein w 1 a +w 1 b= 1 (0≦w 1 a≦ 1, 0≦w 1 b≦ 1), w 2 a +w 2 b= 1 (0≦w 2 a≦ 1, 0≦w 2 b≦ 1), f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal, f 3 represents the first intermediate processed image signal, and f 4 represents the second intermediate processed image signal.

It is preferred that the weighting factors w 1 a, w 1 b, w 2 a and w 2 b in the formulae (5) and (6) be changed depending on the original image signal f in . For example, in the case of an original image signal f in which is set so that the density is increased as the value of the image signal component increases, it is preferred that the weighting factors w 1 a, w 1 b, w 2 a and w 2 b be defined by the following formulae (7) to (10), or by the following formulae (11) to (14).

w 1 a= 1−( cα+t )/ C   (7)

w 1 b =( cα+t )/ C   (8)

w 2 a =( cα+t )/ C   (9)

w 2 b= 1−( cα+t )/ C   (10)

wherein

t=(|f 2 −f in |)/{(|f 1 −f in |)(|f 2 −f in |)},

α=f in /f 0 ,

f in =an original image signal,

f 0 =a reference value,

c=a constant, and

C=a constant.

w 1 a={c (1−α)+ t )}/ C   (11)

w 1 b= 1−{ c (1−α)+ t )}/ C   (12)

w 2 a= 1−( cα+t )/ C   (13)

w 2 b =( cα+t )/ C   (14)

wherein

t=(|f 2 −f in |)/{(|f 1 −f in |)(|f 2 −f in |)},

α=f in /f 0 ,

f in =an original image signal,

f 0 =a reference value,

c=a constant, and

C=a constant.

That is, according to the formulae (7) to (10), or the formulae (11) to (14), the weighting factors w 1 a and w 1 b for the first intermediate processed image signal f 3 are set so that the weight on the granularity-suppressed image signal f 1 becomes larger as the image signal component of the original image signal f in becomes smaller, and the weighting factors w 2 a and w 2 b for the second intermediate processed image signal f 4 are set so that the weight on the sharpness-enhanced image signal f 2 becomes larger as the image signal component of the original image signal f in becomes smaller.

When the weighting factors are set in this manner, in the first intermediate processed image signal f 3 which is selected as the final processed image signal f out for a flat density portion according to formula (4), the granularity is more suppressed at a high density portion than a low density portion. Since the granularity is generally bad at a high density portion in an image, the granularity can be effectively suppressed by more suppressing the granularity at a high density portion.

The weighting factors w 1 a, w 1 b, w 2 a and w 2 b set according to formulae (11) to (14) are preferably applied to a region where the granularity is very bad. In this case, it is preferred that the constants c and C be set in a range c≦C<(c+1) or c<C≦(c+1) in a region where the values of the image signal components of the original image signal f in are relatively small so that w 1 a >w 1 b and w 2 a >w 2 b are satisfied, i.e., so that the weight on the granularity-suppressed image signal f 1 is larger than that on the sharpness-enhanced image signal f 2 in the region for both the intermediate processed image signals f 3 and f 4 (provided that the first intermediate processed image signal f 3 is higher than the second intermediate processed image signal f 4 in the degree of granularity suppression).

The reference value f 0 , which determines α in the formulae (7) to (14), may be, for instance, a maximum value of the image signal components of the original image signal f in (1023 in the case where the original image signal f in is expressed in 10 bits) or 90% of the maximum value. Further it is preferred that the constant C is larger than the constant c.

Formula (15) represents processing in which the image signal component of the first intermediate processed image signal f 3 is taken as the image signal component of the final processed image signal f out when the difference (|f in −f 1 |) between the image signal components of the original image signal f in and the granularity-suppressed image signal f 1 is not smaller than a predetermined threshold value Th, the image signal component of the third intermediate processed image signal f 5 is taken as the image signal component of the final processed image signal f out when the difference (|f in −f 1 |) is smaller than the threshold value Th and the image signal components of both the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 are not smaller than the image signal component of the original image signal f in (k≧0), and the image signal component of the second intermediate processed image signal f 4 is taken as the image signal component of the final processed image signal f out when the difference (|f in −f 1 |) is smaller than the threshold value Th, the image signal component of one of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 is larger than that of the original image signal f in , and the image signal component of the other of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 is smaller than that of the original image signal f in (k<0).

In this case, the first intermediate processed image signal f 3 is generated as an image signal having a sharpness between that of the original image signal f in and that of the sharpness-enhanced image signal f 2 .

According to formula (15), the first intermediate processed image signal f 3 which is substantially equivalent in sharpness to the original image signal f in in which the granularity is not suppressed or to the sharpness-enhanced image signal f 2 in which the sharpness is relatively high, is selected for image portions such as an image of catchlights (external light reflected at a pupil) where the difference (|f in −f 1 |) is larger than the threshold value Th is selected, whereby the image portions such as the image of catchlights can be left there and the image can be rich in expression.

It is preferred that the second intermediate processed image signal f 4 be higher in the degree of granularity suppression than the third intermediate processed image signal f 5 and the third intermediate processed image signal f 5 be higher in the degree of sharpness enhancement than the second intermediate processed image signal f 4 . At portions such as an edge portion of an image where the density abruptly changes, the indicator value k is generally not smaller than 0 (k>0), and accordingly by selecting the third intermediate processed image signal f 5 which is higher than the second intermediate processed image signal f 4 in the degree of sharpness enhancement, the edge portion can be prevented from becoming unsharp. On the other hand, at the other portions (e.g., flat density portions), the indicator value k is generally smaller than 0 (k<0) and the granularity is apt to be remarked, and accordingly, by selecting the second intermediate processed image signal f 4 which is higher than the third intermediate processed image signal f 5 in the degree of granularity suppression, the granularity can be suppressed.

It is preferred that the first to third intermediate processed image signals f 3 , f 4 and f 5 be obtained by weighting the granularity-suppressed image signal f 1 (or the original image signal f in ) and the sharpness-enhanced image signal f 2 with weighting factors w 3 a, w 3 b, w 4 a, w 4 b, w 5 a and w 5 b, and adding the weighted image signal f 1 (or the weighted original image signal f in ) and the weighted image signal f 2 as represented by the following formulae (16) to (18).

f 3 = w 3 a·f 1 + w 3 b·f 2   (16)

f 4 = w 4 a·f 1 + w 4 b·f 2   (17)

f 5 = w 5 a·f 1 + w 5 b·f 2   (18)

wherein w 3 a +w 3 b= 1 (0≦w 3 a≦ 1, 0≦w 3 b≦ 1), w 4 a +w 4 b= 1 (0≦w 4 a≦ 1, 0≦w 4 b≦ 1), w 5 a +w 5 b= 1 (0≦w 5 a≦ 1, 0≦w 5 b≦ 1), f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal, f 3 represents the first intermediate processed image signal, f 4 represents the second intermediate processed image signal, and f 5 represents the third intermediate processed image signal.

It is preferred that the weighting factors w 4 a, w 4 b, w 5 a and w 5 b in the formulae (16) to (18) be changed depending on the original image signal f in . For example, in the case of an original image signal f in which is set so that the density is increased as the value of the image signal component increases, it is preferred that the weighting factors w 4 a, w 4 b, w 5 a and w 5 b be defined by the following formulae (19) to (22), or by the following formulae (23) to (26).

w 4 a= 1−( cα+t )/ C   (19)

w 4 b =( cα+t )/ C   (20)

w 5 a =( cα+t )/ C   (21)

w 5 b= 1−( cα+t )/ C   (22)

wherein

t=(|f 2 −f in |)/{(|f 1 −f in |)(|f 2 −f in |)},

α=f in /f 0 ,

f in =an original image signal,

f 0 =a reference value,

c=a constant, and

C=a constant.

w 4 a={c (1−α)+ t )}/ C   (23)

w 4 b= 1−{ c (1−α)+ t )}/ C   (24)

w 5 a= 1−( cα+t )/ C   (25)

w 5 b =( cα+t )/ C   (26)

wherein

t=(|f 2 −f in |)/{(|f 1 −f in |)(|f 2 −f in |)},

α=f in /f 0 ,

f in =an original image signal,

f 0 =a reference value,

c=a constant, and

C=a constant.

That is, according to the formulae (19) to (22), or the formulae (23) to (26), the weighting factors w 4 a and w 4 b for the second intermediate processed image signal f 4 are set so that the weight on the granularity-suppressed image signal f 1 becomes larger as the image signal component of the original image signal f in becomes smaller, and the weighting factors w 5 a and w 5 b for the third intermediate processed image signal f 5 are set so that the weight on the sharpness-enhanced image signal f 2 becomes larger as the image signal component of the original image signal f in becomes smaller.

When the weighting factors are set in this manner, in the second intermediate processed image signal f 4 which is selected as the final processed image signal f out for a flat density portion according to formula (15), the granularity is more suppressed at a high density portion than a low density portion. Since the granularity is generally bad at a high density portion in an image, the granularity can be effectively suppressed by more suppressing the granularity at a high density portion.

The weighting factors w 4 a, w 4 b, w 5 a and w 5 b set according to formulae (23) to (26) are preferably applied to a region (e.g., extremely underexposed portion) where the granularity is very bad. In this case, it is preferred that the constants c and C be set in a range c≦C<(c+1) or c<C≦(c+1) in a region where the values of the image signal components of the original image signal f in are relatively small so that w 4 a >w 4 b and w 5 a >w 5 b are satisfied, i.e., so that the weight on the granularity-suppressed image signal f 1 is larger than that on the sharpness-enhanced image signal f 2 in the region for both the intermediate processed image signals f 4 and f 5 (provided that the second intermediate processed image signal f 4 is higher than the third intermediate processed image signal f 5 in the degree of granularity suppression).

The reference value f 0 , which determines a in the formulae (19) to (26), may be, for instance, a maximum value of the image signal components of the original image signal f in (1023 in the case where the original image signal f in is expressed in 10 bits) or 90% of the maximum value. Further it is preferred that the constant C is larger than the constant c.

As the granularity suppressing image processing, there can be used, for instance, a median filter processing, hysteresis smoothing processing, noise removing processing by repetition, and granularity suppressing processing (smoothing) using morphology operation. As the sharpness enhancement image processing, there can be used, for instance, unsharp masking processing, high frequency band enhancement filter processing, and processing in which an image signal is separated into a low frequency component, an intermediate frequency component and a high frequency component, the high frequency component is enhanced while the intermediate frequency component is suppressed, and the processed high frequency component, the processed intermediate frequency component and the low frequency component are combined into an image signal (Japanese Unexamined Patent Publication No. 9(1997)-22460). However a combination of the granularity suppressing processing using morphology operation and the processing in which an image signal is separated into a low frequency component, an intermediate frequency component and a high frequency component, the high frequency component is enhanced while the intermediate frequency component is suppressed, and the processed high frequency component, the processed intermediate frequency component and the low frequency component are combined into an image signal (Japanese Unexamined Patent Publication No. 9(1997)-22460) is optimal.

Though the morphology operation is generally developed as a set theory in an N-dimensional space, it is often applied to images which are two-dimensional spaces. See, for instance, Japanese Unexamined Patent Publication Nos. 8(1996)-272961, 9(1997)-248291 and 9(1997)-91421. The morphology operation will be briefly described on a light and shade image by way of example.

Considering a light and shade image as a space where a point (x, y) has a height corresponding to a density value f(x, y), and a one-dimensional function f(x) corresponding a cross-section of the space will be discussed.

It is assumed that structure element g employed in the morphology operation is a symmetric function symmetrical about the origin as represented by the following formula (28) and is 0 in value in its domain G which is defined by the following formula (29).

g 5 ( X )= g (− X )  (28)

G={−m, −m+ 1, . . . , −1, 0, 1, . . . , m− 1, m}   (29)

At this time, the basic of the morphology operation is very simple as represented by the following formula (30) to (33).

dilation;[ f⊕G 5 ]( i )=max{ f ( i−m ), . . . , f ( i ), . . . , f ( i+m )}  (30)

erosion;[ f⊖G 5 ]( i )=min{ f ( i−m ), . . . , f ( i ), . . . , f ( i+m )}  (31)

opening; f g =( f⊖g 5 )⊕ g   (32)

closing; f g =( f⊕g 5 )⊖ g   (33)

The dilation processing is for searching the maximum value in a range of ±m (a value determined depending on structure element B and corresponding the mask size in FIGS. 15A to 15 D) about an object pixel (See, FIG. 15 A), while the erosion processing is for searching the minimum value in the range of ±m about the object pixel (See, FIG. 15 B). The opening processing is for performing the dilation processing after the erosion processing, that is, for searching the value maximum after search of the minimum value, and the closing processing is for performing the erosion processing after the dilation processing, that is, for searching the minimum value after search of the maximum value.

In other words, the opening processing smoothes the density curve f(X) from the low density side and suppresses density projection portions (portions higher in density than a surrounding portion) at which the density fluctuates within a range which is spatially narrower than the mask size 2m (FIG. 15 C). To the contrast, the closing processing smoothes the density curve f(X) from the high density side and suppresses density recession portions (portions lower in density than a surrounding portion) at which the density fluctuates within a range which is spatially narrower than the mask size 2m (FIG. 15 D).

In a high-density high-signal-level image signal, the signal level increases as the density increases whereas in a high-brightness high-signal-level image signal, the signal level increases as the brightness increases. Accordingly, the dilation processing for the high-density high-signal-level image signal is equivalent to the erosion processing for the high-brightness high-signal-level image signal (FIG. 15 B), the erosion processing for the high-density high-signal-level image signal is equivalent to the dilation processing for the high-brightness high-signal-level image signal (FIG. 15 A), the opening processing for the high-density high-signal-level image signal is equivalent to the closing processing for the high-brightness high-signal-level image signal (FIG. 15 D), and the closing processing for the high-density high-signal-level image signal is equivalent to the opening processing for the high-brightness high-signal-level image signal (FIG. 15 C).

By carrying out the opening processing or the closing processing by the morphology operation on an image signal representing an original image, granularity of the image (noise in the image signal) can be suppressed or removed. (“Morphology” by Obata, CORONA)

When granularity suppressing processing using morphology operation is employed in the image processing method of the present invention, it is preferred that granularity suppressing defined by the following formula (27) be employed. $$ f1 = { f opn ⁢   ⁢ ( when ⁢   ⁢ f cls = f in ) f cls ⁢   ⁢ ( when ⁢   ⁢ f opn = f in ) ( f opn + f cls ) / 2 ⁢   ⁢ ( when ⁢   ⁢ f cls ≠ f in ,   ⁢ and ⁢   ⁢ f opn ≠ f in ) ( 27 )

wherein

f 1 : granularity-suppressed image signal,

f cls : closing image signal,

f opn : opening image signal,

f in : original image signal.

When granularity suppressing processing using morphology operation is employed, portions such as an edge portion of an image where the density abruptly changes are substantially perfectly reserved (kept as they are), and accordingly, (f 1 −f in )) in formulae (1), (4) and (15) is substantially constantly not smaller than 0 and the indicator value k is not smaller than 0.

The sharpness enhancement processing by enhancing high frequency components of the image while suppressing intermediate frequency components disclosed in Japanese Unexamined Patent Publication No. 9(1997)-22460will be briefly described, hereinbelow.

In this processing, an image signal is separated into a low frequency component, an intermediate frequency component and a high frequency component, enhancement/suppression processing in which the high frequency component is enhanced and the intermediate frequency component is suppressed is carried out, and the processed high frequency component, the processed intermediate frequency component and the low frequency component are combined into a final processed image signal f out .

The low frequency component, intermediate frequency component and high frequency component are those distributed as shown in FIG. 16 . The intermediate frequency component is distributed with a peak near ⅓ of a Nyquist frequency of output for reproducing the final processed image signal f out as a visible image, the low frequency component is distributed with a peak at a frequency at which the Nyquist frequency of the output becomes 0, and the high frequency component is distributed with a peak at the Nyquist frequency of the output. The sum of the low frequency component, the intermediate frequency component and the high frequency component is 1 at each frequency.

In this processing, it is preferred that brightness components be extracted from the separated high and intermediate frequency components and said enhancement/suppression processing and said combination processing be carried out on the basis of only the brightness components.

Further, in this processing, it is preferred that a particular color region of the image be extracted and the intermediate component fraction corresponding to the particular color region be further suppressed in the enhancement/suppression processing.

Further, in this processing, it is preferred that an evaluation value of the intermediate frequency component and/or the high frequency component be obtained after the image signal is separated into the low frequency component, the intermediate frequency component and the high frequency component, and the intermediate frequency component for a pixel the evaluation value for which is smaller than a predetermined threshold value be more suppressed in the enhancement/suppression processing than the intermediate frequency component for a pixel the evaluation value for which is larger than the predetermined threshold value. In this case, it is further preferred that the high frequency component for the pixel the evaluation value for which is smaller than the predetermined threshold value be less enhanced in the enhancement/suppression processing than the high frequency component for the pixel the evaluation value for which is larger than the predetermined threshold value.

The “evaluation value” as used here means, for instance, at least one of correlation values between twos of red, green and blue, a value of local dispersion of the frequency components of the image signal.

Further, it is preferred that the evaluation value be at least one of correlation values between twos of red, green and blue of the intermediate frequency component and/or the high frequency component for the corresponding pixels.

The evaluation value may be processed with a median filter and the enhancement/suppression processing may be performed on the basis of comparison between the processed evaluation value and said threshold value. Further, the evaluation value of the intermediate frequency component and/or the high frequency component may be calculated on the basis of different colors of red, green and blue.

In granularity suppression in the enhancement/suppression processing of Japanese Unexamined Patent Publication No. 9(1997)-22460, continuity between adjacent pixels in a real image is not taken into account as described above. Accordingly, it is difficult to recognize the granularity suppression effect of the enhancement/suppression processing. To the contrast, the granularity suppression processing based on the morphology operation is processing on a real image space using isolation of granularity (continuity of noise between adjacent pixels is low). Accordingly, when an image signal is processed by this method, continuity between adjacent pixels is improved and the degree of suppression of granularity comes to be more easy to visually recognize.

It is preferred that the original image signal be so-called raw data, data as read out from a recording medium such as a print or a photographic film by a scanner or the like. However the original image signal may be an image signal which has been subjected to gradation processing, color correction, shading correction and/or the like. Further the original image signal need not be an image signal read out by a scanner or the like but may be an image signal such as directly taken out from a digital camera.

In accordance with another aspect of the present invention, there is provided an image processing system for carrying out the image processing method of the present invention. That is, the image processing system of the present invention is for processing an original image signal made up of a plurality of image signal components each representing a pixel of an original image and obtaining a final processed image signal, and comprises,

a granularity suppressing means which carries out processing for suppressing granularity on the original image signal and obtains a granularity-suppressed image signal,

a sharpness enhancing means which carries out processing for enhancing sharpness on the original image signal and obtains sharpness-enhanced image signal,

an intermediate processed image signal generating means which generates at least one intermediate processed image signal by operation based on at least the granularity-suppressed image signal and the sharpness-enhanced image signal, and

a final processed image signal generating means which obtains a final processed image signal by taking, as the image signal component for each pixel of the final processed image signal, the image signal component for the corresponding pixel of an image signal selected from the original image signal and the intermediate processed image signal(s) according to an indicator value calculated on the basis of the granularity-suppressed image signal, the sharpness-enhanced image signal and the original image signal.

As the granularity suppressing image processing to be carried out by the granularity suppressing means, there can be used, for instance, a median filter processing, hysteresis smoothing processing, noise removing processing by repetition, and granularity suppressing processing using morphology operation. As the sharpness enhancement image processing to be carried out by the sharpness enhancing means, there can be used, for instance, unsharp masking processing, high frequency band enhancement filter processing, and processing in which an image signal is separated into a low frequency component, an intermediate frequency component and a high frequency component, the high frequency component is enhanced while the intermediate frequency component is suppressed, and the processed high frequency component, the processed intermediate frequency component and the low frequency component are combined into an image signal (Japanese Unexamined Patent Publication No. 9(1997)-22460). However a combination of the granularity suppressing processing using morphology operation and the processing in which an image signal is separated into a low frequency component, an intermediate frequency component and a high frequency component, the high frequency component is enhanced while the intermediate frequency component is suppressed, and the processed high frequency component, the processed intermediate frequency component and the low frequency component are combined into an image signal (Japanese Unexamined Patent Publication No. 9(1997)-22460) is optimal.

When the granularity suppressing processing using morphology operation is employed as the granularity suppressing image processing to be carried out by the granularity suppressing means, it is preferred that the granularity suppressing processing be that defined by the aforesaid formula (27).

The operation for generating at least one intermediate processed image signal by the intermediate processed image signal generating means may be selected from a plurality of kinds of preset operations according to the relation with the original image signal, the kind of the image and/or the position of the pixel, or may be input from the external for each processing.

Specifically, when one intermediate processed image signal f 3 is to be generated, the operation is preferably that defined by the aforesaid formula (2) which is based on the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 , or that defined by the following formula (3) which is based on the granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in .

It is preferred that the final processed image generating means obtains a final processed image signal by taking the image signal component of one of the intermediate processed image signal f 3 and the original image signal f in as the image signal component for each pixel of the final processed image signal f out according to the aforesaid formula (1).

It is preferred that when two or more intermediate processed image signals f 3 , f 4 , . . . are to be obtained, the operation defined by the aforesaid formulae (5) and (6), or (16) to (18) be applied.

It is preferred that the weighting factors w 1 a, w 1 b, w 2 a and w 2 b in the formulae (5) and (6) and the weighting factors w 4 a, w 4 b, w 5 a and w 5 b in the formulae (16) to (18) be changed depending on the original image signal f in . For example, in the case of an original image signal f in which is set so that the density is increased as the value of the image signal component increases, it is preferred that the weighting factors w 1 a, w 1 b, w 2 a and w 2 b and the weighting factors w 4 a, w 4 b, w 5 a and w 5 b be defined by the aforesaid formulae (7) to (10), (11) to (14), or (23) to (26).

Further it is preferred that the final processed image signal generating means obtains a final processed image signal f out by taking the image signal component of one of the two intermediate processed image signals f 3 and f 4 as the image signal component for each pixel of the final processed image signal f out according to the indicator value by the processing represented by the aforesaid formula (4), or obtains a final processed image signal f out by taking the image signal component of one of the three intermediate processed image signals f 3 , f 4 and f 5 as the image signal component for each pixel of the final processed image signal f out according to the indicator value by the processing represented by the aforesaid formula (15).

It is possible to record the steps of the image processing method of the present invention on a computer-readable recording medium as program for causing a computer to perform the steps.

That is, it is possible to provide a computer-readable recording medium on which the steps of carrying out processing for suppressing granularity and processing for enhancing sharpness separately on the original image signal to obtain a granularity-suppressed image signal and a sharpness-enhanced image signal, obtaining at least one intermediate processed image signal by operation based on at least the granularity-suppressed image signal and the sharpness-enhanced image signal, and taking the image signal component of one of the intermediate processed image signal(s) and the original image signal as the image signal component for each pixel of the final processed image signal according to the indicator value calculated on the basis of the granularity-suppressed image signal, the sharpness-enhanced image signal and the original image signal are recorded as program for causing a computer to perform the steps.

The intermediate processed image signal to be obtained may be one. In this case, the single intermediate processed image signal may be obtained by the operation processing defined by any one of the aforesaid formulae (1), (2) and (3) though the operation processing defined by the aforesaid formula (2) is preferred.

Two different intermediate processed image signals may be obtained. In this case, the two intermediate processed image signal may be obtained by the operation processing defined by the aforesaid formula (4), or by the operation processing defined by the aforesaid formulae (5) and (6). In the case where the operation processing defined by the aforesaid formulae (5) and (6) is employed, it is preferred that the weighting factors w 1 a, w 1 b, w 2 a and w 2 b in the formulae (5) and (6) be defined by the aforesaid formulae (7) to (10) or (11) to (14).

Three different intermediate processed image signals may be obtained. In this case, the three intermediate processed image signal may be obtained by the operation processing defined by the aforesaid formula (15), or by the operation processing defined by the aforesaid formulae (16) to (18). In the case where the operation processing defined by the aforesaid formulae (16) to (18) is employed, it is preferred that the weighting factors w 4 a, w 4 b, w 5 a and w 5 b in the formulae (17) and (18) be defined by the aforesaid formulae (19) to (22) or (23) to (26).

It is preferred that the steps of carrying out processing for suppressing granularity and processing for enhancing sharpness involve the granularity suppressing processing using morphology operation and the processing in which an image signal is separated into a low frequency component, an intermediate frequency component and a high frequency component, the high frequency component is enhanced while the intermediate frequency component is suppressed, and the processed high frequency component, the processed intermediate frequency component and the low frequency component are combined into an image signal.

It is further preferred that the granularity suppressing processing using morphology operation is processing defined by the aforesaid formula (27).

As can be understood from the description above, in accordance with the present invention, since the image signal component of one of the intermediate processed image signal(s) and the original image signal is taken as the image signal component for each pixel of the final processed image signal according to an indicator value calculated on the basis of the granularity-suppressed image signal, the sharpness-enhanced image signal and the original image signal, both the granularity suppression and the sharpness enhancement can be more effectively realized as compared with various conventional methods or the systems.

That is, since the intermediate processed image signal is obtained by processing the image signal components of the granularity-suppressed image signal and the sharpness-enhanced image signal as well as the original image signal, as required, in one-to-one correspondence, the intermediate processed image signal is effectively suppressed with the granularity without deteriorating the sharpness at edge portions or the like of the image. Further since the image signal component of one of the intermediate processed image signal(s) and the original image signal is taken as the image signal component for each pixel of the final processed image signal according to the indicator value which is calculated on the basis of the granularity-suppressed image signal, the sharpness-enhanced image signal and the original image signal and is indicative of the features of kinds of portions of the image, the granularity of the image further precisely suppressed without deteriorating the sharpness at edge portions or the like of the image.

Further when a combination of the granularity suppressing processing using morphology operation and the processing disclosed in Japanese Unexamined Patent Publication No. 9(1997)-22460 in which an image signal is separated into a low frequency component, an intermediate frequency component and a high frequency component, the high frequency component is enhanced while the intermediate frequency component is suppressed, and the processed high frequency component, the processed intermediate frequency component and the low frequency component are combined into an image signal, the following result can be further expected.

That is, when the operation processing represented by the aforesaid formula (3) is employed on an image signal where the granularity suppression has not been carried out on sharpness enhanced portion, strong sharpness enhancing processing results in increase in the granularity to cancel the granularity suppressing processing which has been carried out on the original image signal to obtain the granularity-suppressed image signal. Accordingly, in the case where the operation processing represented by the aforesaid formula (3) is employed, it is preferred that the sharpness enhancing processing disclosed in Japanese Unexamined Patent Publication No. 8(1997)-22460 where the granularity can be suppressed to a certain extent be employed. Further when the sharpness at edge portions is deteriorated by the granularity suppressing processing, this result affects the result of the sharpness enhancing processing, and accordingly, the granularity suppressing processing using morphology operation which can suppress the granularity without deteriorating the sharpness at edge portions is preferred.

Thus it should be appreciated that by employing granularity suppressing processing, such as the granularity suppressing processing using morphology operation which can reserve the sharpness at edge portion, as the processing for suppressing the granularity, and employing the sharpness enhancing processing such as disclosed in Japanese Unexamined Patent Publication No. 8(1997)-22460 where the granularity can be suppressed to a certain extent as the processing for enhancing the sharpness, a more preferable intermediate processed image signal can be obtained.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart for illustrating the flow of processing in an image processing method in accordance with a first embodiment of the present invention,

FIG. 2 is a schematic view showing an image processing system for carrying out the method shown in FIG. 1 ,

FIG. 3 is a flow chart for illustrating the flow of processing in an image processing method in accordance with a second embodiment of the present invention,

FIG. 4 is a schematic view showing an image processing system for carrying out the method shown in FIG. 3 ,

FIG. 5 is a view showing in detail the arrangement of the granularity suppressing means,

FIG. 6 is a view showing in detail the arrangement of the sharpness enhancing means,

FIG. 7 is a flow chart for illustrating the flow of processing in an image processing method in accordance with a third embodiment of the present invention,

FIG. 8 is a schematic view showing an image processing system for carrying out the method shown in FIG. 7 ,

FIG. 9 is a flow chart for illustrating the flow of processing in an image processing method in accordance with a fourth embodiment of the present invention,

FIG. 10 is a schematic view showing an image processing system for carrying out the method shown in FIG. 9 ,

FIG. 11 is a flow chart for illustrating the flow of processing in an image processing method in accordance with a fifth embodiment of the present invention,

FIG. 12 is a schematic view showing an image processing system for carrying out the method shown in FIG. 11 ,

FIG. 13 is a flow chart for illustrating the flow of processing in an image processing method in accordance with a sixth embodiment of the present invention,

FIG. 14 is a schematic view showing an image processing system for carrying out the method shown in FIG. 13 ,

FIGS. 15A to 15 D are views for illustrating a basic operation of morphology operation, and

FIG. 16 is a graph showing the distributions of the low frequency, intermediate frequency and high frequency components.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

In FIG. 2 , an image processing system for carrying out the image processing method in accordance with a first embodiment of the present invention shown in FIG. 1 comprises a granularity suppressing means 11 which receives an original image signal f in representing an original image and carries out processing for suppressing granularity on the original image signal f in , thereby obtaining a granularity-suppressed image signal f 1 , a sharpness enhancing means 12 which receives the original image signal f in and carries out processing for enhancing sharpness on the original image signal f in , thereby obtaining a sharpness-enhanced image signal f 2 , an intermediate processed image signal generating means 20 which generates an intermediate processed image signal f 3 by carrying out operation g(f 1 , f 2 )=f 3 based on the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 on pixels one by one with the image signal components of the image signals f 1 and f 2 in one-to-one correspondence with each other, and a final processed image signal generating means 30 which obtains a final processed image signal f out by taking the image signal component of one of the intermediate processed image signal f 3 and the original image signal f in as the image signal component for each pixel of the final processed image signal f out according to an indicator value k(f 1 , f 2 , f in ) calculated on the basis of the granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in .

For example, the operation to be performed by the intermediate processed image signal generating means 20 for obtaining the intermediate processed image signal f 3 (=g(f 1 , f 2 )) may be weighted addition defined by the following formula (2).

f 3 = w 1 −· f 1 + w 2 · f 2   (2)

wherein w 1 +w 2 =1 (0<w 1 <1, 0<w 2 <1), f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal and f 3 represents the intermediate processed image signal. That is, g(f 1 , f 2 )=w 1 ·f 1 +w 2 ·f 2 .

The values of the weighting factors w 1 and w 2 are set to different values by pixel by pixel according to, for instance, the values of the image signal components of the original image signal f in . Specifically, for the image signal components for pixels in the flat density portion (an area of the image where change of the density of brightness is less) where the granularity is more apt to be remarked than the sharpness, the weighting factor w 1 on the granularity-suppressed image signal f 1 is set larger than the weighting factor w 2 on the sharpness-enhanced image signal f 2 . To the contrast, for the image signal components for pixels in the edge portion (an area of the image where change of the density or brightness is sharp) where the sharpness is more apt to be remarked than the granularity, the weighting factor w 2 on the sharpness-enhanced image signal f 2 is set larger than the weighting factor w 1 on the granularity-suppressed image signal f 1 .

In this particular embodiment, as the granularity suppression processing by the granularity suppressing means 11 , median filter processing is employed, and as the sharpness enhancing processing, unsharp masking (USM) processing is employed.

The final processed image signal generating means 30 takes the image signal component of one of the intermediate processed image signal f 3 and the original image signal f in as the image signal component of the final processed image signal f out according to, for instance, the image signal components of the granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in , e.g, whether the indicator value k(f 1 , f 2 , f in )=(f 1 −f in )·(f 2 −f in ) is positive or negative, and comprises a determination section 31 which determines whether the indicator value k(f 1 , f 2 , f in )=(f 1 −f in )·(f 2 −f in ) is positive or negative and a selection section 32 which selects the image signal component of one of the intermediate processed image signal f 3 and the original image signal f in as the image signal component of the final processed image signal f out on the basis of determination by the determination section 31 as represented by the following formula (1). $$ f out = { f in ⁢   ⁢ ( when ⁢   ⁢ ( f1 - f in ) · ( f2 - f in ) < 0 ) , f3 ⁢   ⁢ ( when ⁢   ⁢ ( f1 - f in ) · ( f2 - f in ) ≧ 0 ) ( 1 )

wherein f in represents the original image signal, f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal, and f 3 represents the intermediate processed image signal.

Operation of the image processing system of this embodiment will be described, hereinbelow.

First the original image signal f in is separately input into the granularity suppressing means 11 and the sharpness enhancing means 12 . The original image signal f in has an edge portion where fluctuation in density is relatively large and a flat density portion where fluctuation in density is relatively small. High frequency noise is superimposed on these density distributions, and the high frequency noise is apt to be remarked especially in the flat density portion.

The granularity suppressing means 11 carries out median filer processing on the original image signal f in and calculates the granularity-suppressed image signal f 1 . The sharpness enhancing means 12 carries out USM processing on the original image signal f in and calculates the sharpness-enhanced image signal f 2 . The granularity-suppressed image signal f 1 is an image signal where the granularity of the original image signal f in is suppressed and smoothed and the sharpness-enhanced image signal f 2 is an image signal where the large density fluctuation portion of the original image signal f in is enhanced.

The granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 are input into the intermediate processed image signal generating means 20 , and the intermediate processed image signal generating means 20 generates the intermediate processed image signal f 3 by carrying out the weighted addition according to formula (2) on the basis of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 . For the image signal components for pixels in the flat density portion where the granularity is more apt to be remarked than the sharpness, the weighting factor w 1 on the granularity-suppressed image signal f 1 is set larger than the weighting factor w 2 on the sharpness-enhanced image signal f 2 , and for the image signal components for pixels in the edge portion where the sharpness is more apt to be remarked than the granularity, the weighting factor w 2 on the sharpness-enhanced image signal f 2 is set larger than the weighting factor w 1 on the granularity-suppressed image signal f 1 . Accordingly, in the intermediate processed image signal f 3 , the granularity of the original image signal f in is suppressed in the flat density portion where the granularity is more apt to be remarked and the sharpness of the original image signal f in is enhanced in the edge portion where the sharpness is more apt to be remarked. That is, the intermediate processed image signal f 3 is an image signal where the suppression of the granularity and the enhancement of the sharpness are consistent with each other.

The intermediate processed image signal f 3 thus obtained is input into the final processed image signal generating means 30 . The original image signal f in , the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 are also input into the final processed image signal generating means 30 . The determination section 31 of the final processed image signal generating means 30 determines whether the indicator value k(f 1 , f 2 , f in )=(f 1 −f in )·(f 2 −f in ) is positive or negative and inputs the result of the determination into the selection section 32 .

The selection section 32 selects the image signal component of one of the intermediate processed image signal f 3 and the original image signal f in as the image signal component of the final processed image signal f out on the basis of determination by the determination section 31 according to formula (1), that is, when the indicator value k is not smaller than 0, the selection section 32 outputs the image signal component of the intermediate processed image signal f 3 as the image signal component of the final processed image signal f out and when the indicator value k is smaller than 0, the selection section 32 outputs the image signal component of the original image signal f in as the image signal component of the final processed image signal f out .

At portions such as an edge portion of original image where the density abruptly changes, the indicator value k is generally not smaller than 0 (k≧0), and accordingly the intermediate processed image signal f 3 where the sharpness is higher than the original image signal f in is selected. On the other hand, at the other portions (e.g., flat density portions), the indicator value k is generally smaller than 0 (k<0), and accordingly, the original image signal f in is selected since sufficient granularity suppressing effect cannot be obtained with the intermediate processed image signal f 3 which is a weighted average of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 . In this case, by selecting the original image signal f in , a granularity-suppressed final processed image signal f out can be obtained. The image represented by the final processed image signal f out thus obtained is an image signal where the granularity is further suppressed without deterioration in the sharpness of the edge portions and the like as compared with the intermediate processed image signal f 3 .

As can be understood from the description above, with the image processing system of this embodiment, both the granularity suppression and the sharpness enhancement can be more effectively realized as compared with various conventional methods or the systems.

FIG. 3 is a flow chart for illustrating the flow of processing in an image processing method in accordance with a second embodiment of the present invention, and FIG. 4 is a schematic view showing an image processing system for carrying out the method shown in FIG. 3 . The image processing system shown in FIG. 4 is basically the same as that shown in FIG. 2 except that the original image signal f in is one of a red original image signal R in , a green original image signal G in and a blue original image signal B in which are red, green and blue components of a color image signal representing a color image such as taken by use of a digital still camera or photoelectrically read out from a color print or a color film, the intermediate processed image signal generating means 20 ′ generates an intermediate processed image signal f 3 by carrying out operation g(f 1 , f 2 , f in )=f 3 based on the granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in the granularity suppressing means 11 ′ carries out granularity suppression processing using morphology operation on the original image signal f in , and the sharpness enhancing means 12 ′ carries out sharpness enhancement processing in which the original image signal f in is separated into a low frequency component, an intermediate frequency component and a high frequency component, the high frequency component is enhanced while the intermediate frequency component is suppressed, and the processed high frequency component, the processed intermediate frequency component and the low frequency component are combined into an image signal.

For the purpose of simplicity, each of the red original image signal R in , the green original image signal G in and the blue original image signal B in will be considered to be an original image signal f in , each of granularity-suppressed image signals R 1 , G 1 and B 1 respectively obtained from the red original image signal R in , the green original image signal G in and the blue original image signal B in will be considered to be a granularity-suppressed image signal f 1 , each of sharpness-enhanced image signals R 2 , G 2 and B 2 respectively obtained from the red original image signal R in , the green original image signal G in and the blue original image signal B in will be considered to be a sharpness-enhanced image signal f 2 , each of intermediate processed image signals G 3 , R 3 and B 3 will be considered to be an intermediate processed image signal f 3 , and each of the final processed image signals R out , G out and B out will be considered to be a final processed image signal f out .

The operation to be performed by the intermediate processed image signal generating means 20 ′ for obtaining the intermediate processed image signal f 3 (=g(f 1 , f 2 , f in )) may be weighted addition defined by the following formula (3).

  f 3 ={| f 1 − f in |( f 1 − f in )+| f 2 − f in |( f 2 − f in )}/{| f 1 − f in |+|f 2 − f in |}+f in   (3)

wherein f in represents the original image signal, f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal and f 3 represents the intermediate processed image signal.

In accordance with formula (3), the image signal components of the sharpness-enhanced image signal f 2 are taken as the image signal components of the intermediate processed image signal f 3 for pixels where the image signal components of the granularity-suppressed image signal f 1 are equal to those of the original image signal f in , the image signal components of the granularity-suppressed image signal f 1 are taken as the image signal components of the intermediate processed image signal f 3 for pixels where the image signal components of the sharpness-enhanced image signal f 2 are equal to those of the original image signal f in , and values obtained by weighting the differences between the image signal components of the granularity-suppressed image signal f 1 and the original image signal f in and between the image signal components of the sharpness-enhanced image signal f 2 and the original image signal f in and adding the weighted differences are taken as the image signal components of the intermediate processed image signal f 3 for pixels where the image signal components of neither of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 are equal to those of the original image signal f in . With this operation, for the image signal components for pixels in the flat density portion (an area of the image where change of the density or brightness is less) where the granularity is more apt to be remarked than the sharpness, the image signal components of the granularity-suppressed image signal f 1 is more weighted and accordingly the granularity can be suppressed. To the contrast, for the image signal components for pixels in the edge portion (an area of the image where change of the density or brightness is sharp) where the sharpness is more apt to be remarked than the granularity, the image signal components of the sharpness-enhanced image signal f 2 is more weighted and accordingly the sharpness can be enhanced.

As shown in FIG. 5 , the granularity suppressing means 11 ′ comprises an opening processing means 11 a which carries out opening processing on the input original image signal f in (R in , G in , B in ) and obtains an opening image signal f opn (R opn , G opn , B opn ), a closing processing means 11 b which carries out closing processing on the input original image signal f in (R in , G in , B in ) and obtains a closing image signal f cls (R cls , G cls , B cls ), and a morphology MIX means 11 c which obtains a granularity-suppressed image signal f 1 (R 1 , G 1 , B 1 ) according to the following formula (27) on the basis of the opening image signal f opn , the closing image signal f cls and the original image signal f in . $$ f1 = { f1 = f opn ⁢   ⁢ ( when ⁢   ⁢ f cls = f in ) f cls ⁢   ⁢ ( when ⁢   ⁢ f opn = f in ) f1 = ( f opn + f cls ) / 2 ⁢   ⁢ ( when ⁢   ⁢ f cls ≠ f in , f opn ≠ f in ) ( 27 )

wherein

f 1 : granularity-suppressed image signal,

f cls : closing image signal,

f opn : opening image signal,

f in : original image signal.

The sharpness enhancing means 12 ′ comprises, as shown in FIG. 6 , a 9 pixels×9 pixels low-pass filter (LPF) 12 a which carries out filtering processing on the input original image signal f in (R in , G in , B in ) and extracts low frequency component (R L , G L , B L ) of the original image signal f in , an operator 12 b which subtracts the low frequency component (R L , G L , B L ) from the original image signal (R in , G in , B in ) and extracts intermediate to high frequency component (R MH , G MH , B MH ), a brightness signal extracting means (MTX) 12 c which extracts a brightness component Y MH from the intermediate to high frequency component (R MH , G MH , B MH ), a 5 pixels×5 pixels low-pass filter 12 g which carries out filtering processing on the brightness component Y MH and extracts an intermediate frequency component Y M of the brightness component Y MH , an operator 12 h which subtracts the intermediate frequency component Y M from the brightness component Y MH and extracts a high frequency component Y H of the brightness component Y MH , a variable gain amplifier 12 i into which the intermediate frequency component Y M is input, a variable gain amplifier 12 j into which the high frequency component Y H is input, a correlation value calculating means 12 d which calculates the correlation value ε among red, green and blue of the intermediate to high frequency component (R MH , G MH , B MH ), a lookup table (LUT) 12 e in which the correlation value ε is related to the gain M to the intermediate frequency component Y M , a lookup table (LUT) 12 f in which the correlation value ε is related to the gain H to the high frequency component Y H , an operator 12 k which combines the intermediate frequency component Y′ M output from the variable gain amplifier 12 i with the high frequency component Y′ H output from the variable gain amplifier 12 j into a combined intermediate to high frequency component Y′ MH , and an operator 12 m which combines the combined intermediate to high frequency component Y′ MH with the low frequency component (R L , R G , R B ) into a sharpness-enhanced image signal f 2 (R 2 , G 2 , B 2 ).

The LUT 12 e and the LUT 12 f are set so that the gain M for a given correlation value ε is smaller than the gain H for the given correlation value ε up to a predetermined value of the correlation value ε.

The granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in are input into the intermediate processed image signal generating means 20 ′, and the intermediate processed image signal generating means 20 ′ generates the intermediate processed image signal f 3 (R 3 , G 3 , B 3 ) by carrying out the operation according to formula (3).

The intermediate processed image signal f 3 thus obtained is input into the final processed image signal generating means 30 together with the original image signal f in , the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 . The determination section 31 of the final processed image signal generating means 30 determines whether the indicator value k(f 1 , f 2 , f in )=(f 1 −f in )·(f 2 −f in ) is positive or negative and inputs the result of the determination into the selection section 32 .

The selection section 32 selects the image signal component of one of the intermediate processed image signal f 3 and the original image signal f in as the image signal component of the final processed image signal f out on the basis of determination by the determination section 31 according to formula (1), that is, when the indicator value k is not smaller than 0, the selection section 32 outputs the image signal component of the intermediate processed image signal f 3 as the image signal component of the final processed image signal f out and when the indicator value k is smaller than 0, the selection section 32 outputs the image signal component of the original image signal f in as the image signal component of the final processed image signal f out .

As can be understood from the description above, with the image processing system of this embodiment, both the granularity suppression and the sharpness enhancement can be more effectively realized as compared with various conventional methods or the systems.

Though, in the embodiments described above, the intermediate processed image signal generating means 20 generates a single intermediate processed image signal f 3 and the final processed image signal generating means 30 selects as the final processed image signal f out one of the intermediate processed image signal f 3 and the original image signal f in according to the indicator value k, the method and system of the present invention need not be limited to such a form. For example, the intermediate processed image signal generating means 20 may generate two or more different intermediate processed image signals f 3 , f 4 , . . . and the final processed image signal generating means 30 may select as the final processed image signal f out one of the intermediate processed image signals f 3 , f 4 , . . . or one of the intermediate processed image signals f 3 , f 4 , . . . and the original image signal f in .

FIG. 7 is a flow chart for illustrating the flow of processing in an image processing method in accordance with a third embodiment of the present invention, and FIG. 8 is a schematic view showing an image processing system for carrying out the method shown in FIG. 7 .

In FIG. 8 , the image processing system for carrying out the image processing method in accordance with the third embodiment of the present invention shown in FIG. 7 comprises a granularity suppressing means 11 which receives an original image signal f in representing an original image and carries out processing for suppressing granularity on the original image signal f in , thereby obtaining a granularity-suppressed image signal f 1 , a sharpness enhancing means 12 which receives the original image signal f in and carries out processing for enhancing sharpness on the original image signal f in , thereby obtaining a sharpness-enhanced image signal f 2 , an intermediate processed image signal generating means 20 which generates two different intermediate processed image signals f 3 and f 4 by carrying out operation g(f 1 , f 2 )=f 3 , h(f 1 , f 2 )=f 4 based on the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 on pixels one by one with the image signal components of the image signals f 1 and f 2 in one-to-one correspondence with each other, and a final processed image signal generating means 30 which obtains a final processed image signal f out by taking the image signal component of one of the first and second intermediate processed image signals f 3 and f 4 as the image signal component for each pixel of the final processed image signal f out according to an indicator value k(f 1 , f 2 , f in ) calculated on the basis of the granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in .

The operations to be performed by the intermediate processed image signal generating means 20 for obtaining the intermediate processed image signals f 3 (=g(f 1 , f 2 )) and f 4 (=h(f 1 , f 2 )) are weighted additions defined by the following formulae (5) and (6).

f 3 = w 1 a·f 1 + w 1 b·f 2   (5)

f 4 = w 2 a·f 1 + w 2 b·f 2   (6)

wherein w 1 a +w 1 b= 1 (0≦w 1 a≦ 1, 0≦w 1 b≦ 1), w 2 a +w 2 b= 1 (0≦w 2 a≦ 1, 0≦w 2 b≦ 1), f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal, f 3 represents the first intermediate processed image signal, and f 4 represents the second intermediate processed image signal.

That is, g(f 1 , f 2 )=w 1 a ·f 1 +w 1 b ·f 2 , h(f 1 , f 2 )=w 2 a ·f 1 +w 2 b ·f 2 .

The values of the weighting factors w 1 a, w 1 b, w 2 a and w 2 b are set to different values by pixel by pixel according to, for instance, the values of the image signal components of the original image signal f in . Specifically, the weighting factors w 1 a, w 1 b, w 2 a and w 2 b are set according to the following formulae (7) to (10), or the following formulae (11) to (14). According to the formulae (7) to (10), or the formulae (11) to (14), the weighting factors w 1 a and w 2 b are increased as the image signal component of the original image signal f in becomes smaller and the weight on the granularity-suppressed image signal f 1 becomes larger than that on the sharpness-enhanced image signal f 2 in the first intermediate processed image signal f 3 and the weight on the sharpness-enhanced image signal f 2 becomes larger than that on the granularity-suppressed image signal f 1 in the second intermediate processed image signal f 4 as the image signal component of the original image signal f in becomes smaller, whereby the first intermediate processed image signal f 3 becomes higher in the degree of suppression of the granularity than the second intermediate processed image signal f 4 .

To the contrast, according to the formulae (7) to (10), or the formulae (11) to (14), the weighting factors w 1 b and w 2 a are increased as the image signal component of the original image signal f in becomes larger and the weight on the sharpness-enhanced image signal f 2 becomes larger than that on the granularity-suppressed image signal f 1 in the first intermediate processed image signal f 3 and the weight on the granularity-suppressed image signal f 1 becomes larger than that on the sharpness-enhanced image signal f 2 in the second intermediate processed image signal f 4 as the image signal component of the original image signal f in becomes larger, whereby the second intermediate processed image signal f 4 becomes higher in the degree of suppression of the granularity than the first intermediate processed image signal f 3 .

The weighting factors w 1 a, w 1 b, w 2 a and w 2 b set according to formulae (11) to (14) are preferably applied to a region where the granularity is very bad.

w 1 a= 1−( cα+t )/ C   (7)

w 1 b =( cα+t )/ C   (8)

w 2 a =( cα+t )/ C   (9)

w 2 b= 1−( cα+t )/ C   (10)

wherein

t=(|f 2 −f in |)/{(|f 1 −f in |)(|f 2 −f in |)},

α=f in /f 0 ,

f in =an original image signal,

f 0 =a reference value,

c=a constant, and

C=a constant.

w 1 a={c (1−α)+ t )}/ C   (11)

w 1 b= 1−{ c (1−α)+ t )}/ C   (12)

w 2 a= 1−( cα+t )/ C   (13)

w 2 b =( cα+t )/ C   (14)

wherein

t=(|f 2 −f in |)/{(|f 1 −f in |)(|f 2 −f in |)},

α=f in /f 0 ,

f in =an original image signal,

f 0 =a reference value,

c=a constant, and

C=a constant.

In this particular embodiment, as the granularity suppression processing by the granularity suppressing means 11 , median filter processing is employed, and as the sharpness enhancing processing, unsharp masking (USM) processing is employed.

The final processed image signal generating means 30 takes the image signal component of one of the first and second intermediate processed image signals f 3 and f 4 as the image signal component of the final processed image signal f out according to, for instance, the image signal components of the granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in , e.g., whether the indicator value k(f 1 , f 2 , f in )=(f 1 −f in )·(f 2 −f in ) is positive or negative, and comprises a determination section 31 which determines whether the indicator value k(f 1 , f 2 , f in )=(f 1 −f in )·(f 2 −f in ) is positive or negative and a selection section 32 which selects the image signal component of one of the first and second intermediate processed image signal f 3 and f 4 as the image signal component of the final processed image signal f out on the basis of determination by the determination section 31 as represented by the following formula (4). $$ f out = { f3 ⁢   ⁢ ( when ⁢   ⁢ ( f1 - f in ) · ( f2 - f in ) < 0 ) , f4 ⁢   ⁢ ( when ⁢   ⁢ ( f1 - f in ) · ( f2 - f in ) ≧ 0 ) ( 4 )

wherein f in represents the original image signal, f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal, f 3 represents the first intermediate processed image signal, and f 4 represents the second intermediate processed image signal.

Operation of the image processing system of this embodiment will be described, hereinbelow.

First the original image signal f in is separately input into the granularity suppressing means 11 and the sharpness enhancing means 12 . The original image signal f in has an edge portion where fluctuation in density is relatively large and a flat density portion where fluctuation in density is relatively small. High frequency noise is superimposed on these density distributions, and the high frequency noise is apt to be remarked especially in the flat density portion.

The granularity suppressing means 11 carries out median filer processing on the original image signal f in and calculates the granularity-suppressed image signal f 1 . The sharpness enhancing means 12 carries out USM processing on the original image signal f in and calculates the sharpness-enhanced image signal f 2 . The granularity-suppressed image signal f 1 is an image signal where the granularity of the original image signal f in is suppressed and smoothed and the sharpness-enhanced image signal f 2 is an image signal where the large density fluctuation portion of the original image signal f in is enhanced.

The granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 are input into the intermediate processed image signal generating means 20 , and the intermediate processed image signal generating means 20 generates the intermediate processed image signals f 3 and f 4 by carrying out the weighted addition according to formulae (5) and (6) and the formulae (7) to (10) or the formulae (11) to (14) on the basis of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 .

When the weighting factors are calculated according to formulae (7) to (10), the weight on the granularity-suppressed image signal f 1 becomes larger than that on the sharpness-enhanced image signal f 2 in the first intermediate processed image signal f 3 and the weight on the sharpness-enhanced image signal f 2 becomes larger than that on the granularity-suppressed image signal f 1 in the second intermediate processed image signal f 4 as the image signal component of the original image signal f in becomes smaller.

To the contrast, according to the formulae (7) to (10), the weight on the sharpness-enhanced image signal f 2 becomes larger than that on the granularity-suppressed image signal f 1 in the first intermediate processed image signal f 3 and the weight on the granularity-suppressed image signal f 1 becomes larger than that on the sharpness-enhanced image signal f 2 in the second intermediate processed image signal f 4 as the image signal component of the original image signal f in becomes larger.

When the weighting factors are calculated according to formulae (11) to (14), the weight on the granularity-suppressed image signal f 1 is larger than that on the sharpness-enhanced image signal f 2 in either of the first and second intermediate processed image signals f 3 and f 4 in a region where the image signal components of the original image signal f in are relatively small and the first intermediate processed image signal f 3 is higher in the degree of suppression of the granularity than the second intermediate processed image signal f 4 . Accordingly, the weighting factors calculated according to formulae (11) to (14) are preferably applied to a region where the granularity is very bad.

The intermediate processed image signals f 3 and f 4 thus obtained are input into the final processed image signal generating means 30 . The original image signal f in , the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 are also input into the final processed image signal generating means 30 . The determination section 31 of the final processed image signal generating means 30 determines whether the indicator value k(f 1 , f 2 , f in )=(f 1 −f in )·(f 2 −f in ) is positive or negative and inputs the result of the determination into the selection section 32 .

The selection section 32 selects the image signal component of one of the intermediate processed image signal f 3 and the second intermediate processed image signal f 4 as the image signal component of the final processed image signal f out on the basis of determination by the determination section 31 according to formula (4), that is, when the indicator value k is not smaller than 0, the selection section 32 outputs the image signal component of the second intermediate processed image signal f 4 as the image signal component of the final processed image signal f out and when the indicator value k is smaller than 0, the selection section 32 outputs the image signal component of the first intermediate processed image signal f 3 as the image signal component of the final processed image signal f out .

At portions such as an edge portion of original image where the density abruptly changes, the indicator value k is generally not smaller than 0 (k≧0), and accordingly the second intermediate processed image signal f 4 is selected. Since the second intermediate processed image signal f 4 is obtained by weighting the sharpness-enhanced image signal f 2 more than the granularity-suppressed image signal f 1 on the side of the edge portion where the density is lower than the other side according to formulae (9) and (10) or (13) and (14), the second intermediate processed image signal f 4 may be considered to be a sharpness-enhanced image signal where the edge portion of the image is enhanced.

On the other hand, at the other portions (e.g., flat density portions), the indicator value k is generally smaller than 0 (k<0), and accordingly, the first intermediate processed image signal f 3 is selected. As can be seen from formulae (7) and (8) or (11) and (12), though being an image signal which is relatively low in the degree of granularity suppression in a low density region where the granularity is less apt to be remarked, the first intermediate processed image signal f 3 may be considered to be an image signal in which the weight on the granularity-suppressed image signal f 1 is larger than that on the sharpness-enhanced image signal f 2 in a high density region where the granularity is more apt to be remarked and the granularity of the image is effectively suppressed.

As can be understood from the description above, with the image processing system of this embodiment, both the granularity suppression and the sharpness enhancement can be more effectively realized according to the features of portions of the image (whether an edge portion or a flat density portion) as compared with various conventional methods or the systems, and at the same time the degree of granularity suppression can be properly controlled according to the density of the image.

FIG. 13 is a flow chart for illustrating the flow of processing in an image processing method in accordance with a fourth embodiment of the present invention, and FIG. 14 is a schematic view showing an image processing system for carrying out the method shown in FIG. 13 . The image processing system shown in FIG. 14 is basically the same as that shown in FIG. 12 except that the original image signal f in is one of a red original image signal R in , a green original image signal G in and a blue original image signal B in which are red, green and blue components of a color image signal representing a color image such as taken by use of a digital still camera or photoelectrically read out from a color print or a color film, the intermediate processed image signal generating means 20 ′ generates first and second intermediate processed image signals f 3 and f 4 by carrying out operation g(f 1 , f 2 , f in )=f 3 , and h(f 1 , f 2 , f in )=f 4 based on the granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in , the granularity suppressing means 11 ′ carries out granularity suppression processing using morphology operation on the original image signal f in , and the sharpness enhancing means 12 ′ carries out sharpness enhancement processing in which the original image signal f in is separated into a low frequency component, an intermediate frequency component and a high frequency component, the high frequency component is enhanced while the intermediate frequency component is suppressed, and the processed high frequency component, the processed intermediate frequency component and the low frequency component are combined into an image signal.

For the purpose of simplicity, each of the red original image signal R in , the green original image signal G in and the blue original image signal B in will be considered to be an original image signal f in , each of granularity-suppressed image signals R 1 , G 1 and B 1 respectively obtained from the red original image signal R in , the green original image signal G in and the blue original image signal B in will be considered to be a granularity-suppressed image signal f 1 , each of sharpness-enhanced image signals R 2 , G 2 and B 2 respectively obtained from the red original image signal R in , the green original image signal G in and the blue original image signal B in will be considered to be a sharpness-enhanced image signal f 2 , each of intermediate processed image signals G 3 , R 3 and B 3 will be considered to be an intermediate processed image signal f 3 , each of intermediate processed image signals G 4 , R 4 and B 4 will be considered to be an intermediate processed image signal f 4 , each of intermediate processed image signals G 5 , R 5 and B 5 will be considered to be an intermediate processed image signal f 5 , and each of the final processed image signals R out , G out and B out will be considered to be a final processed image signal f out .

The operation to be performed by the intermediate processed image signal generating means 20 ′ for obtaining the intermediate processed image signals f 3 (=g(f 1 , f 2 , f in )), f 4 (=h(f 1 , f 2 , f in )) and f 5 (=j(f 1 , f 2 , f in )) may be weighted addition defined by formulae (16) to (18) with the weighting factors defined by formula (19) to (22) or (23) to (26).

The granularity suppressing means 11 ′, as shown in FIG. 5 , comprises an opening processing means 11 a which carries out opening processing on the input original image signal f in (R in , G in , B in ) and obtains an opening image signal f opn (R opn , G opn , B opn ), a closing processing means 11 b which carries out closing processing on the input original image signal f in (R in , G in , B in ) and obtains a closing image signal f cls (R cls , G cls , B cls ), and a morphology MIX means 11 c which obtains a granularity-suppressed image signal f 1 (R 1 , G 1 , B 1 ) according to the aforesaid formula (27) on the basis of the opening image signal f opn , the closing image signal f cls and the original image signal f in .

The sharpness enhancing means 12 ′ comprises, as shown in FIG. 6 , a low-pass filter (LPF) 12 a, an operator 12 b which extracts an intermediate to high frequency component (R MH , G MH , B MH ), a brightness signal extracting means (MTX) 12 c which extracts a brightness component Y MH from the intermediate to high frequency component (R MH , G MH , B MH ), a low-pass filter 12 g, an operator 12 h which extracts a high frequency component Y H of the brightness component Y H , a variable gain amplifier 12 i into which the intermediate frequency component Y M is input, a variable gain amplifier 12 j into which the high frequency component Y H is input, a correlation value calculating means 12 d which calculates the correlation value ε among red, green and blue of the intermediate to high frequency component (R MH , G MH , B MH ), a lookup table (LUT) 12 e in which the correlation value ε is related to the gain M to the intermediate frequency component Y M , a lookup table (LUT) 12 f in which the correlation value ε is related to the gain H to the high frequency component Y H , an operator 12 k which combines the intermediate frequency component Y′ M output from the variable gain amplifier 12 i with the high frequency component Y′ H output from the variable gain amplifier 12 j into a combined intermediate to high frequency component Y′ MH , and an operator 12 m which combines the combined intermediate to high frequency component Y′ MH with the low frequency component (R L , R G , R B ) into a sharpness-enhanced image signal f 2 (R 2 , G 2 , B 2 ).

The LUT 12 e and the LUT 12 f are set so that the gain M for a given correlation value ε is smaller than the gain H for the given correlation value ε up to a predetermined value of the correlation value ε.

The granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in are input into the intermediate processed image signal generating means 20 ′, and the intermediate processed image signal generating means 20 ′ generates the first and second intermediate processed image signals f 3 (R 3 , G 3 , B 3 ) and f 4 (R 4 , G 4 , B 4 ) by carrying out the operation according to formulae (16) to (18).

The intermediate processed image signals f 3 and f 4 thus obtained are input into the final processed image signal generating means 30 together with the original image signal f in , the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 . The determination section 31 of the final processed image signal generating means 30 determines whether the indicator value k(f 1 , f 2 , f in )=(f 1 −f in )·(f 2 −f in ) is positive or negative and inputs the result of the determination into the selection section 32 .

The selection section 32 selects the image signal component of one of the first and second intermediate processed image signals f 3 and f 4 as the image signal component of the final processed image signal f out on the basis of determination by the determination section 31 according to formula (4), that is, when the indicator value k is not smaller than 0, the selection section 32 outputs the image signal component of the second intermediate processed image signal f 4 as the image signal component of the final processed image signal f out and when the indicator value k is smaller than 0, the selection section 32 outputs the image signal component of the first intermediate processed image signal f 3 as the image signal component of the final processed image signal f out .

Like the final processed image signal f out obtained in the third embodiment, the final processed image signal f out output from the image processing system of this embodiment is an image signal in which the sharpness is enhanced in a region such as an edge portion where the density charge is sharp while the granularity is suppressed to relatively small extent in a low density region in the regions other than the edge portion and the granularity is effectively suppressed in a high density region in the regions other than the edge portion.

As can be understood from the description above, with the image processing system of this embodiment, both the granularity suppression and the sharpness enhancement can be more effectively realized according to the features of portions of the image (whether an edge portion or a flat density portion) as compared with various conventional methods or the systems, and at the same time the degree of granularity suppression can be properly controlled according to the density of the image.

Though, in the embodiments described above, the intermediate processed image signal generating means 20 generates first and second intermediate processed image signals f 3 and f 4 the final processed image signal generating means 30 selects as the final processed image signal f out one of the intermediate processed image signals f 3 and f 4 and the original image signal f in according to the indicator value k, in the fifth embodiment shown in FIGS. 11 and 12 , the intermediate processed image signal generating means 20 generates three different intermediate processed image signals f 3 , f 4 and f 5 , and the final processed image signal generating means 30 may select as the final processed image signal f out one of the intermediate processed image signals f 3 , f 4 , f 5 and the original image signal f in .

FIG. 11 is a flow chart for illustrating the flow of processing in an image processing method in accordance with the fifth embodiment of the present invention, and FIG. 12 is a schematic view showing an image processing system for carrying out the method shown in FIG. 11 .

In FIG. 12 , the image processing system for carrying out the image processing method in accordance with the fifth embodiment of the present invention comprises a granularity suppressing means 11 which receives an original image signal f in representing an original image and carries out processing for suppressing granularity on the original image signal f in , thereby obtaining a granularity-suppressed image signal f 1 , a sharpness enhancing means 12 which receives the original image signal f in and carries out processing for enhancing sharpness on the original image signal f in , thereby obtaining a sharpness-enhanced image signal f 2 , an intermediate processed image signal generating means 20 which generates three different (first to third) intermediate processed image signals f 3 , f 4 and f 5 by carrying out operation g(f 1 , f 2 )=f 3 , h(f 1 , f 2 )=f 4 , j(f 1 , f 2 ) based on the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 on pixels one by one with the image signal components of the image signals f 1 and f 2 in one-to-one correspondence with each other, and a final processed image signal generating means 30 which obtains a final processed image signal f out by taking the image signal component of one of the first to third intermediate processed image signals f 3 , f 4 and f 5 as the image signal component for each pixel of the final processed image signal f out according to an indicator value k(f 1 , f 2 , f in ) calculated on the basis of the granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in .

The operations to be performed by the intermediate processed image signal generating means 20 for obtaining the intermediate processed image signals f 3 (=g(f 1 , f 2 )), f 4 (=h(f 1 , f 2 )) and f 5 (j(f 1 , f 2 )) are weighted additions defined by the following formulae (16) to (18).

f 3 = w 3 a·f 1 + w 3 b·f 2   (16)

f 4 = w 4 a·f 1 + w 4 b·f 2   (17)

f 5 = w 5 a·f 1 + w 5 b·f 2   (18)

wherein w 3 a +w 3 b= 1 (0≦w 3 a≦ 1, 0≦w 3 b≦ 1), w 4 a +w 4 b= 1 (0≦w 4 a≦ 1, 0≦w 4 b≦ 1), w 5 a +w 5 b= 1 (0≦w 5 a≦ 1, 0≦w 5 b≦ 1), f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal, f 3 represents the first intermediate processed image signal, f 4 represents the second intermediate processed image signal, and f 5 represents the third intermediate processed image signal.

That is, g(f 1 , f 2 )=w 3 a ·f 1 +w 3 b ·f 2 , h(f 1 , f 2 )=w 4 a ·f 1 +w 4 b ·f 2 , j(f 1 , f 2 )=w 5 a ·f 1 +w 5 b ·f 2 .

The values of the weighting factors w 4 a, w 4 b, w 5 a and w 5 b are set to different values by pixel by pixel according to, for instance, the values of the image signal components of the original image signal f in . Specifically, the weighting factors w 4 a, w 4 b, w 5 a and w 5 b are set according to the following formulae (19) to (22), or the following formulae (23) to (26). According to the formulae (19) to (22), or the formulae (23) to (26), the weighting factors w 4 a and w 5 b are increased as the image signal component of the original image signal f in becomes smaller and the weight on the granularity-suppressed image signal f 1 becomes larger than that on the sharpness-enhanced image signal f 2 in the second intermediate processed image signal f 4 and the weight on the sharpness-enhanced image signal f 2 becomes larger than that on the granularity-suppressed image signal f 1 in the third intermediate processed image signal f 5 as the image signal component of the original image signal f in becomes smaller, whereby the second intermediate processed image signal f 4 becomes higher in the degree of suppression of the granularity than the third intermediate processed image signal f 5 .

To the contrast, according to the formulae (19) to (22), or the formulae (23) to (26), the weighting factors w 4 b and w 5 a are increased as the image signal component of the original image signal f in becomes larger and the weight on the sharpness-enhanced image signal f 2 becomes larger than that on the granularity-suppressed image signal f 1 in the second intermediate processed image signal f 4 and the weight on the granularity-suppressed image signal f 1 becomes larger than that on the sharpness-enhanced image signal f 2 in the third intermediate processed image signal f 5 as the image signal component of the original image signal f in becomes larger, whereby the third intermediate processed image signal f 5 becomes higher in the degree of suppression of the granularity than the second intermediate processed image signal f 4 . In this case, the first intermediate processed image signal f 3 is generated as an image signal having a sharpness between that of the original image signal f in and that of the sharpness-enhanced image signal f 2 .

The weighting factors w 4 a, w 4 b, w 5 a and w 5 b set according to formulae (23) to (26) are preferably applied to a region where the granularity is very bad (e.g., extremely underexposed portion).

w 4 a= 1−( cα+t )/ C   (19)

w 4 b =( cα+t )/ C   (20)

w 5 a =( cα+t )/ C   (21)

w 5 b= 1−( cα+t )/ C   (22)

wherein

t=(|f 2 −f in |)/{(|f 1 −f in |)(|f 2 −f in |)},

α=f in /f 0 ,

f in =an original image signal,

f 0 =a reference value,

c=a constant, and

C=a constant.

w 4 a={c (1−α)+ t )}/ C   (23)

w 4 b= 1−{ c (1−α)+ t )}/ C   (24)

w 5 a= 1−( cα+t )/ C   (25)

w 5 b =( cα+t )/ C   (26)

wherein

t=(|f 2 −f in |)/{(|f 1 −f in |)(|f 2 −f in |)},

α=f in /f 0 ,

f in =an original image signal,

f 0 =a reference value,

c=a constant, and

C=a constant.

In this particular embodiment, as the granularity suppression processing by the granularity suppressing means 11 , median filter processing is employed, and as the sharpness enhancing processing, unsharp masking (USM) processing is employed.

The final processed image signal generating means 30 takes the image signal component of one of the first to third intermediate processed image signals f 3 , f 4 and f 5 as the image signal component of the final processed image signal f out according to, for instance, the image signal components of the granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in , e.g, according to whether the indicator value k(f 1 , f 2 , f in )=(f 1 −f in )·(f 2 −f in ) is positive or negative and the value of |f in −f 1 |, as represented by the following formula (15) and comprises a determination section 31 which determines whether the indicator value k(f 1 , f 2 , f in )=(f 1 −f in )·(f 2 −f in ) is positive or negative and whether |f in −f 1 | is larger than a predetermined constant Th and a selection section 32 which selects the image signal component of one of the first to third intermediate processed image signal f 3 , f 4 and f 5 as the image signal component of the final processed image signal f out on the basis of determination by the determination section 31 . $$ f out = { f3 ⁢   ⁢ ( when ⁢   ⁢ &LeftBracketingBar; fin - f1 &RightBracketingBar; ≧ Th ) f4 ⁢   ⁢ ( when ⁢   ⁢ &LeftBracketingBar; fin - f1 &RightBracketingBar; < Th ⁢   ⁢ and ⁢   ⁢ ( f1 - fin ) · ( f2 - fin ) < 0 ) ⁢   f5 ⁢   ⁢ ( when ⁢   ⁢ &LeftBracketingBar; fin - f1 &RightBracketingBar; < Th ⁢   ⁢ and ⁢   ⁢ ( f1 - fin ) · ( f2 - fin ) ≧ 0 ) ( 15 )

wherein f in represents the original image signal, f 1 represents the granularity-suppressed image signal, f 2 represents the sharpness-enhanced image signal, f 3 represents the first intermediate processed image signal, f 4 represents the second intermediate processed image signal, f 5 represents the third intermediate processed image signal, and Th represents a constant.

Operation of the image processing system of this embodiment will be described, hereinbelow.

First the original image signal f in is separately input into the granularity suppressing means 11 and the sharpness enhancing means 12 . The original image signal f in has an edge portion where fluctuation in density is relatively large and a flat density portion where fluctuation in density is relatively small. High frequency noise is superimposed on these density distributions, and the high frequency noise is apt to be remarked especially in the flat density portion.

The granularity suppressing means 11 carries out median filer processing on the original image signal f in and calculates the granularity-suppressed image signal f 1 . The sharpness enhancing means 12 carries out USM processing on the original image signal f in and calculates the sharpness-enhanced image signal f 2 . The granularity-suppressed image signal f 1 is an image signal where the granularity of the original image signal f in is suppressed and smoothed and the sharpness-enhanced image signal f 2 is an image signal where the large density fluctuation portion of the original image signal f in is enhanced.

The granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 are input into the intermediate processed image signal generating means 20 , and the intermediate processed image signal generating means 20 generates the intermediate processed image signals f 3 , f 4 and f 5 by carrying out the weighted addition according to formulae (16) to (18) and the formulae (19) to (22) or the formulae (23) to (26) on the basis of the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 .

The first intermediate processed image signal f 3 is generated as an image signal having a sharpness between that of the original image signal f in and that of the sharpness-enhanced image signal f 2 .

When the weighting factors are calculated according to formulae (19) to (22), the weight on the granularity-suppressed image signal f 1 becomes larger than that on the sharpness-enhanced image signal f 2 in the second intermediate processed image signal f 4 and the weight on the sharpness-enhanced image signal f 2 becomes larger than that on the granularity-suppressed image signal f 1 in the third intermediate processed image signal f 5 as the image signal component of the original image signal f in becomes smaller. To the contrast, according to the formulae (19) to (22), the weight on the sharpness-enhanced image signal f 2 becomes larger than that on the granularity-suppressed image signal f 1 in the second intermediate processed image signal f 4 and the weight on the granularity-suppressed image signal f 1 becomes larger than that on the sharpness-enhanced image signal f 2 in the third intermediate processed image signal f 4 as the image signal component of the original image signal f in becomes larger.

When the weighting factors are calculated according to formulae (23) to (26), the weight on the granularity-suppressed image signal f 1 is larger than that on the sharpness-enhanced image signal f 2 in either of the second and third intermediate processed image signals f 4 and f 5 in a region where the image signal components of the original image signal f in are relatively small and the second intermediate processed image signal f 4 is higher in the degree of suppression of the granularity than the third intermediate processed image signal f 5 . Accordingly, the weighting factors calculated according to formulae (23) to (26) are preferably applied to a region where the granularity is very bad.

The intermediate processed image signals f 3 , f 4 and f 5 thus obtained are input into the final processed image signal generating means 30 . The original image signal f in , the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 are also input into the final processed image signal generating means 30 . The determination section 31 of the final processed image signal generating means 30 determines whether the indicator value k(f 1 , f 2 , f in )=(f 1 −f in )·(f 2 −f in ) is positive or negative and whether |f in −f 1 | is larger than the constant Th and inputs the result of the determination into the selection section 32 .

The selection section 32 selects the image signal component of one of the first to third intermediate processed image signals f 3 , f 4 and f 5 as the image signal component of the final processed image signal f out on the basis of determination by the determination section 31 according to formula (15), that is, when (|f in −f 1 |) is not smaller than the predetermined threshold value Th, the selection section 32 outputs the image signal component of the first intermediate processed image signal f 3 , which relatively high in sharpness as the image signal component of the final processed image signal f out , and when the indicator value k is not smaller than 0 with (|f in −f 1 |) is smaller than the threshold value Th, the selection section 32 outputs the image signal component of the third intermediate processed image signal f 5 as the image signal component of the final processed image signal f out , and when the indicator value k is smaller than 0 with (|f in −f 1 |) is smaller than the threshold value Th, the selection section 32 outputs the image signal component of the second intermediate processed image signal f 4 as the image signal component of the final processed image signal f out .

When the difference (|f in −f 1 |) between the original image signal f in and the granularity-suppressed image signal f 1 is large, for instance, image portions such as an image of catchlights, the first intermediate processed image signal f 3 which is relatively high in sharpness is selected, whereby the image portions such as the image of catchlights can be left there and the image can be rich in expression.

Even if, the difference (|f in −f 1 |) is smaller than the threshold value Th, the third intermediate processed image signal f 5 is selected if the indicator value k is not smaller than 0 (k≧0). Since the third intermediate processed image signal f 5 is obtained by weighting the sharpness-enhanced image signal f 2 more than the granularity-suppressed image signal f 1 on the side of the edge portion where the density is lower than the other side according to formulae (21) and (22) or (25) and (26), the third intermediate processed image signal f 5 may be considered to be a sharpness-enhanced image signal where the edge portion of the image is enhanced.

On the other hand, at portions other than the edge portion and the like (e.g., flat density portions), the indicator value k is generally smaller than 0 (k<0), and accordingly, the second intermediate processed image signal f 4 is selected. As can be seen from formulae (19) and (20) or (23) and (24), though being an image signal which is relatively low in the degree of granularity suppression in a low density region where the granularity is less apt to be remarked, the second intermediate processed image signal f 4 may be considered to be an image signal in which the weight on the granularity-suppressed image signal f 1 is larger than that on the sharpness-enhanced image signal f 2 in a high density region where the granularity is more apt to be remarked and the granularity of the image is effectively suppressed.

As can be understood from the description above, with the image processing system of this embodiment, both the granularity suppression and the sharpness enhancement can be more effectively realized according to the features of portions of the image (whether an edge portion or a flat density portion) as compared with various conventional methods or the systems, and at the same time the degree of granularity suppression can be properly controlled according to the density of the image.

FIG. 13 is a flow chart for illustrating the flow of processing in an image processing method in accordance with a sixth embodiment of the present invention, and FIG. 14 is a schematic view showing an image processing system for carrying out the method shown in FIG. 13 . The image processing system shown in FIG. 14 is basically the same as that shown in FIG. 12 except that the original image signal f in is one of a red original image signal R in , a green original image signal G in and a blue original image signal B in which are red, green and blue components of a color image signal representing a color image such as taken by use of a digital still camera or photoelectrically read out from a color print or a color film, the intermediate processed image signal generating means 20 ′ generates first to third intermediate processed image signals f 3 , f 4 and f 5 by carrying out operation g(f 1 , f 2 , f in )=f 3 , h(f 1 , f 2 , f in )=f 4 and j(f 1 , f 2 , f in )=f 5 based on the granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in , the granularity suppressing means 11 ′ carries out granularity suppression processing using morphology operation on the original image signal f in , and the sharpness enhancing means 12 ′ carries out sharpness enhancement processing in which the original image signal f in is separated into a low frequency component, an intermediate frequency component and a high frequency component, the high frequency component is enhanced while the intermediate frequency component is suppressed, and the processed high frequency component, the processed intermediate frequency component and the low frequency component are combined into an image signal.

For the purpose of simplicity, each of the red original image signal R in , the green original image signal G in and the blue original image signal B in will be considered to be an original image signal f in , each of granularity-suppressed image signals R 1 , G 1 and B 1 respectively obtained from the red original image signal R in , the green original image signal G in and the blue original image signal B in will be considered to be a granularity-suppressed image signal f 1 , each of sharpness-enhanced image signals R 2 , G 2 and B 2 respectively obtained from the red original image signal R in , the green original image signal G in and the blue original image signal B in will be considered to be a sharpness-enhanced image signal f 2 , each of intermediate processed image signals G 3 , R 3 and B 3 will be considered to be an intermediate processed image signal f 3 , and each of the final processed image signals R out , G out and B out will be considered to be a final processed image signal f out .

The operation to be performed by the intermediate processed image signal generating means 20 ′ for obtaining the intermediate processed image signals f 3 (=g(f 1 , f 2 , f in )), f 4 (=h(f 1 , f 2 , f in )) and f 5 (=j(f 1 , f 2 , f in )) are weighted additions defined by formulae (16) to (18) with the weighting factors defined by formula (19) to (22) or (23) to (26).

The granularity suppressing means 11 ′, as shown in FIG. 5 , comprises an opening processing means 11 a which carries out opening processing on the input original image signal f in (R in , G in , B in ) and obtains an opening image signal f opn (R opn , G opn , B opn ), a closing processing means 11 b which carries out closing processing on the input original image signal f in (R in , G in , B in ) and obtains a closing image signal f cls (R cls , G cls , B cls ), and a morphology MIX means 11 c which obtains a granularity-suppressed image signal f 1 (R 1 , G 1 , B 1 ) according to the aforesaid formula (27) on the basis of the opening image signal f opn , the closing image signal f cls and the original image signal f in .

The sharpness enhancing means 12 ′ comprises, as shown in FIG. 6 , a low-pass filter (LPF) 12 a, an operator 12 b which extracts an intermediate to high frequency component (R MH , G MH , B MH ), a brightness signal extracting means (MTX) 12 c which extracts a brightness component Y MH from the intermediate to high frequency component (R MH , G MH , B MH ), a low-pass filter 12 g, an operator 12 h which extracts a high frequency component Y H of the brightness component Y MH , a variable gain amplifier 12 i into which the intermediate frequency component Y M is input, a variable gain amplifier 12 j into which the high frequency component Y H is input, a correlation value calculating means 12 d which calculates the correlation value ε among red, green and blue of the intermediate to high frequency component (R MH , G MH , B MH ), a lookup table (LUT) 12 e in which the correlation value ε is related to the gain M to the intermediate frequency component Y M , a lookup table (LUT) 12 f in which the correlation value ε is related to the gain H to the high frequency component Y H , an operator 12 k which combines the intermediate frequency component Y′ M output from the variable gain amplifier 12 i with the high frequency component Y′ H output from the variable gain amplifier 12 j into a combined intermediate to high frequency component Y′ MH , and an operator 12 m which combines the combined intermediate to high frequency component Y′ MH with the low frequency component (R L , R G , R B ) into a sharpness-enhanced image signal f 2 (R 2 , G 2 , B 2 ).

The LUT 12 e and the LUT 12 f are set so that the gain M for a given correlation value ε is smaller than the gain H for the given correlation value ε up to a predetermined value of the correlation value ε.

The granularity-suppressed image signal f 1 , the sharpness-enhanced image signal f 2 and the original image signal f in are input into the intermediate processed image signal generating means 20 ′, and the intermediate processed image signal generating means 20 ′ generates the first to third intermediate processed image signals f 3 (R 3 , G 3 , B 3 ), f 4 (R 4 , G 4 , B 4 ) and f 45 (R 5 , G 5 , B 5 ) by carrying out the operation according to formulae (16) to (18).

The intermediate processed image signals f 3 , f 4 and f 5 thus obtained are input into the final processed image signal generating means 30 together with the original image signal f in , the granularity-suppressed image signal f 1 and the sharpness-enhanced image signal f 2 . The determination section 31 of the final processed image signal generating means 30 determines whether the indicator value k(f 1 , f 2 , f in )=(f 1 −f in )·(f 2 −f in ) is positive or negative and whether the difference |f in −f 1 | is larger than the threshold value Th, and inputs the result of the determination into the selection section 32 .

The selection section 32 selects the image signal component of one of the first to third intermediate processed image signals f 3 , f 4 and f 5 as the image signal component of the final processed image signal f out on the basis of determination by the determination section 31 according to formula (15), that is, when (|f in −f 1 |) is not smaller than the predetermined threshold value Th, the selection section 32 outputs the image signal component of the first intermediate processed image signal f 3 , which relatively high in sharpness as the image signal component of the final processed image signal f out , and when the indicator value k is not smaller than 0 with (|f in −f 1 |) is smaller than the threshold value Th, the selection section 32 outputs the image signal component of the third intermediate processed image signal f 5 as the image signal component of the final processed image signal f out , and when the indicator value k is smaller than 0 with (|f in −f 1 |) is smaller than the threshold value Th, the selection section 32 outputs the image signal component of the second intermediate processed image signal f 4 as the image signal component of the final processed image signal f out .

Like the final processed image signal f out obtained in the fifth embodiment, the final processed image signal f out obtained in the sixth embodiment is relatively high in sharpness for image portions such as an image of catchlights so that the texture of the image of catchlights is not deteriorated, is enhanced in sharpness for image portions such as an edge portion where the density change is sharp, is relatively low in the granularity for low density image portions other than the edge portion and the like (e.g., flat density portions), and is effectively suppressed with the granularity for high density image portions.

As can be understood from the description above, with the image processing system of this embodiment, both the granularity suppression and the sharpness enhancement can be more effectively realized according to the features of portions of the image (whether an edge portion or a flat density portion) as compared with various conventional methods or the systems, and at the same time the degree of granularity suppression can be properly controlled according to the density of the image.

It is possible to record the steps of the image processing method of the present invention on a computer-readable recording medium as program for causing a computer to perform the steps.

Claims

1. An image processing method of processing an original image signal made up of a plurality of image signal components each representing a pixel of an original image and obtaining a final processed image signal, the method comprising the steps ofcarrying out processing for suppressing granularity and processing for enhancing sharpness separately on the original image signal to obtain a granularity-suppressed image signal and a sharpness-enhanced image signal, obtaining at least one intermediate processed image signal by operation based on at least the granularity-suppressed image signal and the sharpness-enhanced image signal, and taking, as the image signal component for each pixel of the final processed image signal, the image signal component for the corresponding pixel of an image signal selected from the original image signal and the intermediate processed image signal(s) according to an indicator value calculated on the basis of the granularity-suppressed image signal, the sharpness-enhanced image signal and the original image signal.

2. An image processing method as defined in claim 1 in which a single intermediate processed image signal is obtained and, as the image signal component for each pixel of the final processed image signal, the image signal component for the corresponding pixel of one of the intermediate processed image signal and the original image signal is taken according to processing defined by the following formula (1), fout={fin⁢ ⁢(when⁢ ⁢(f1-fin)·(f2-fin)<0),f3⁢ ⁢(when⁢ ⁢(f1-fin)·(f2-fin)≧0)(1)wherein fin represents the original image signal, f1 represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal, and f3 represents the intermediate processed image signal.

3. An image processing method as defined in claim 1 in which a single intermediate processed image signal is obtained and the operation for obtaining the intermediate processed image signal is defined by the following formula (2),f3=w1·f1+w2·f2  (2) wherein w1+w2=1 (0<w1<1, 0<w2<1), f1 represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal and f3 represents the intermediate processed image signal.

4. An image processing method as defined in claim 1 in which a single intermediate processed image signal is obtained and the operation for obtaining the intermediate processed image signal is defined by the following formula (3),f3={|f1−fin|(f1−fin)+|f2−fin|(f2−fin)}/{|f1−fin|+|f2−fin|}+fin  (3) wherein fin represents the original image signal, ff represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal and f3 represents the intermediate processed image signal.

5. An image processing method as defined in claim 1 in which first and second intermediate processed image signals are obtained and, as the image signal component for each pixel of the final processed image signal, the image signal component for the corresponding pixel of an image signal selected from the intermediate processed image signals and the original image signal is taken according to processing defined by the following formula (4), fout={f3⁢ ⁢(when⁢ ⁢(f1-fin)·(f2-fin)<0),f4⁢ ⁢(when⁢ ⁢(f1-fin)·(f2-fin)≧0)(4)wherein fin represents the original image signal, f1 represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal, f3 represents the first intermediate processed image signal, and f4 represents the second intermediate processed image signal.

6. An image processing method as defined in claim 1 in which first and second intermediate processed image signals are obtained and the operation for obtaining the intermediate processed image signal is defined by the following formulae (5) and (6),f3=w1a·f1+w1b·f2  (5) f4=w2a·f1+w2b·f2  (6) wherein w1a+w1b=1 (0≦w1a≦1, 0≦w1b≦1), w2a+w2b=1 (0≦w2a≦1, 0≦w2b≦1), f1 represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal, f3 represents the first intermediate processed image signal, and f4 represents the second intermediate processed image signal.

7. An image processing method as defined in claim 6 in which the weighting factors w1a, w1b, w2a and w2b are defined by the following formulae (7) to (10),w1a=1−(cα+t)/C  (7) w1b=(cα+t)/C  (8) w2a=(cα+t)/C  (9) w2b=1−(cα+t)/C  (10) wherein t=(|f2−fin|)/{(|f1−fin|)(|f2−fin|)}, α=fin/f0, fin represents the original image signal, f0 represents a reference value, c represents a constant, and C represents a constant.

8. An image processing method as defined in claim 6 in which the weighting factors w1a, w1b, w2a and w2b are defined by the following formulae (11) to (14),w1a={c(1−α)+t)}/C  (11) w1b=1−{c(1−α)+t)}/C  (12) w2a=1−(cα+t)/C  (13) w2b=(cα+t)/C  (14) wherein t=(|f2−fin|)/{(|f1−fin|)(|f2−fin|)}, α=fin/f0, fin represents the original image signal, f0 represents a reference value, c represents a constant, and C represents a constant.

9. An image processing method as defined in claim 1 in which first to third intermediate processed image signals are obtained and, as the image signal component for each pixel of the final processed image signal, the image signal component for the corresponding pixel of an image signal selected from the intermediate processed image signals and the original image signal is taken according to processing defined by the following formula (15), fout={f3⁢ ⁢(when⁢ ⁢&LeftBracketingBar;fin-f1&RightBracketingBar;≧Th)f4⁢ ⁢(when⁢ ⁢&LeftBracketingBar;fin-f1&RightBracketingBar;<Th⁢ ⁢and⁢ ⁢(f1-fin)·(f2-fin)<0)⁢ f5⁢ ⁢(when⁢ ⁢&LeftBracketingBar;fin-f1&RightBracketingBar;<Th⁢ ⁢and⁢ ⁢(f1-fin)·(f2-fin)≧0)(15)wherein fin represents the original image signal, f1 represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal, f3 represents the first intermediate processed image signal, f4 represents the second intermediate processed image signal, f5 represents the third intermediate processed image signal, and Th represents a constant.

10. An image processing method as defined in claim 1 in which first to third intermediate processed image signals are obtained and the operation for obtaining the intermediate processed image signals is defined by the following formulae (16) to (18),f3=w3a·f1+w3b·f2  (16) f4=w4a·f1+w4b·f2  (17) f5=w5a·f1+w5b·f2  (18) wherein w3a+w3b=1 (0≦w3a≦1, 0≦w3b≦1), w4a+w4b=1 (0≦w4a≦1, 0≦w4b≦1), w5a+w5b=1 (0≦w5a≦1, 0≦w5b≦1), f1 represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal, f3 represents the first intermediate processed image signal, f4 represents the second intermediate processed image signal, and f5 represents the third intermediate processed image signal.

11. An image processing method as defined in claim 10 in which the weighting factors w4a, w4b, w5a and w5b are defined by the following formulae (19) to (22),w4a=1−(cα+t)/C  (19) w4b=(cα+t)/C  (20) w5a=(cα+t)/C  (21) w5b=1−(cα+t)/C  (22) wherein t=(|f2−fin|)/{(|f1−fin|)(|f2−fin|)}, α=fin/f0, fin represents an original image signal, f0 represents a reference value, c represents a constant, and C represents a constant.

12. An image processing method as defined in claim 10 in which the weighting factors w4a, w4b, w5a and w5b are defined by the following formulae (23) to (26),w4a={c(1−α)+t)}/C  (23) w4b=1−{c(1−α)+t)}/C  (24) w5a=1−(cα+t)/C  (25) w5b=(cα+t)/C  (26) wherein t=(|f2−fin|)/{(|f1−fin|)(|f2−fin|)}, α=fin/f0, fin represents an original image signal, f0 represents a reference value, c represents a constant, and C represents a constant.

13. An image processing method as defined in claim 1 in which the step of carrying out processing for suppressing granularity involves granularity suppressing processing using morphology operation and the step of carrying out processing for enhancing sharpness involves processing in which the original image signal is separated into a low frequency component, an intermediate frequency component and a high frequency component, the high frequency component is enhanced while the intermediate frequency component is suppressed, and the processed high frequency component, the processed intermediate frequency component and the low frequency component are combined into an image signal.

14. An image processing method as defined in claim 13 in which the granularity suppressing processing using morphology operation is defined by the following formula (27), f1={fopn⁢ ⁢(when⁢ ⁢fcls=fin)fcls⁢ ⁢(when⁢ ⁢fopn=fin)(fopn+fcls)/2⁢ ⁢(when⁢ ⁢fcls≠fin, ⁢and⁢ ⁢fopn≠fin)(27)wherein f1 represents the granularity-suppressed image signal, fcls represents a closing image signal, fopn represents an opening image signal, and fin represents the original image signal.

15. An image processing system for processing an original image signal made up of a plurality of image signal components each representing a pixel of an original image and obtaining a final processed image signal, the system comprisinga granularity suppressing means which carries out processing for suppressing granularity on the original image signal and obtains a granularity-suppressed image signal, a sharpness enhancing means which carries out processing for enhancing sharpness on the original image signal and obtains sharpness-enhanced image signal, an intermediate processed image signal generating means which generates at least one intermediate processed image signal by operation based on at least the granularity-suppressed image signal and the sharpness-enhanced image signal, and a final processed image signal generating means which obtains a final processed image signal by taking, as the image signal component for each pixel of the final processed image signal, the image signal component for the corresponding pixel of an image signal selected from the intermediate processed image signals and the original image signal according to an indicator value calculated on the basis of the granularity-suppressed image signal, the sharpness-enhanced image signal and the original image signal.

16. An image processing system as defined in claim 15 in which the intermediate processed image signal generating means generates a single intermediate processed image signal and the final processed image signal generating means takes, as the image signal component for each pixel of the final processed image signal, the image signal component for the corresponding pixel of an image signal selected from the intermediate processed image signals and the original image signal according to processing defined by the following formula (1), fout={fin⁢ ⁢(when⁢ ⁢(f1-fin)·(f2-fin)<0),f3⁢ ⁢(when⁢ ⁢(f1-fin)·(f2-fin)≧0)(1)wherein fin represents the original image signal, f1 represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal, and f3 represents the intermediate processed image signal.

17. An image processing system as defined in claim 15 in which the intermediate processed image signal generating means generates a single intermediate processed image signal according to the operation defined by the following formula (2),f3=w1·f1+w2·f2  (2) wherein w1+w2=1 (0<w1<1, 0<w2<1), f1 represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal and f3 represents the intermediate processed image signal.

18. An image processing system as defined in claim 15 in which the intermediate processed image signal generating means generates a single intermediate processed image signal according to the operation defined by the following formula (3),f3={|f1−fin|(f1−fin)+|f2−fin|(f2−fin)}/{|f1−fin|+|f2−fin|}+fin  (3) wherein fin represents the original image signal, f1 represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal and f3 represents the intermediate processed image signal.

19. An image processing system as defined in claim 15 in which the intermediate processed image signal generating means generated first and second intermediate processed image signals and the final processed image signal generating means takes, as the image signal component for each pixel of the final processed image signal, the image signal component for the corresponding pixel of an image signal selected from the intermediate processed image signals and the original image signal according to processing defined by the following formula (4), fout={f3⁢ ⁢(when⁢ ⁢(f1-fin)·(f2-fin)<0),f4⁢ ⁢(when⁢ ⁢(f1-fin)·(f2-fin)≧0)(4)wherein fin represents the original image signal, f1 represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal, f3 represents the first intermediate processed image signal, and f4 represents the second intermediate processed image signal.

20. An image processing system as defined in claim 15 in which the intermediate processed image signal generating means generates first and second intermediate processed image signals according to the operation defined by the following formulae (5) and (6),f3=w1a·f1+w1b·f2  (5) f4=w2a·f1+w2b·f2  (6) wherein w1a+w1b=1 (0≦w1a≦1, 0≦w1b≦1), w2a+w2b=1 (0≦w2a≦1, 0≦w2b≦1), f1 represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal, f3 represents the first intermediate processed image signal, and f4 represents the second intermediate processed image signal.

21. An image processing system as defined in claim 20 in which the weighting factors w1a, w1b, w2a and w2b are defined by the following formulae (7) to (10),w1a=1−(cα+t)/C  (7) w1b=(cα+t)/C  (8) w2a=(cα+t)/C  (9) w2b=1−(cα+t)/C  (10) wherein t=(|f2−fin|)/{(|f1−fin|)(|f2−fin|)}, α=fin/f0, fin represents the original image signal, f0 represents a reference value, c represents a constant, and C represents a constant.

22. An image processing system as defined in claim 20 in which the weighting factors w1a, w1b, w2a and w2b are defined by the following formulae (11) to (14),w1a={c(1−α)+t)}/C  (11) w1b=1−{c(1−α)+t)}/C  (12) w2a=1−(cα+t)/C  (13) w2b=(cα+t)/C  (14) wherein t=(|f2−fin|)/{(|f1−fin|)(|f2−fin|)}, α=fin/f0, fin represents the original image signal, f0 represents a reference value, c represents a constant, and C represents a constant.

23. An image processing system as defined in claim 15 in which the intermediate processed image signal generating means generates first to third intermediate processed image signals and the final processed image signal generating means takes, as the image signal component for each pixel of the final processed image signal, the image signal component for the corresponding pixel of an image signal selected from the intermediate processed image signals and the original image signal according to the processing defined by the following formula (15), fout={f3⁢ ⁢(when⁢ ⁢&LeftBracketingBar;fin-f1&RightBracketingBar;≧Th)f4⁢ ⁢(when⁢ ⁢&LeftBracketingBar;fin-f1&RightBracketingBar;<Th⁢ ⁢and⁢ ⁢(f1-fin)·(f2-fin)<0)⁢ f5⁢ ⁢(when⁢ ⁢&LeftBracketingBar;fin-f1&RightBracketingBar;<Th⁢ ⁢and⁢ ⁢(f1-fin)·(f2-fin)≧0)(15)wherein fin represents the original image signal, f1 represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal, f3 represents the first intermediate processed image signal, f4 represents the second intermediate processed image signal, f5 represents the third intermediate processed image signal, and Th represents a constant.

24. An image processing system as defined in claim 15 in which the intermediate processed image signal generating means generates first to third intermediate processed image signals according to the operation defined by the following formulae (16) to (18),f3=w3a·f1+w3b·f2  (16)  f4=w4a·f1+w4b·f2  (17)f5=w5a·f1+w5b·f2  (18) wherein w3a+w3b=1 (0≦w3a≦1, 0≦w3b≦1), w4a+w4b=1 (0≦w4a≦1, 0≦w4b≦1), w5a+w5b=1 (0≦w5a≦1, 0≦w5b≦1), f1 represents the granularity-suppressed image signal, f2 represents the sharpness-enhanced image signal, f3 represents the first intermediate processed image signal, f4 represents the second intermediate processed image signal, and f5 represents the third intermediate processed image signal.

25. An image processing system as defined in claim 24 in which the weighting factors w4a, w4b, w5a and w5b are defined by the following formulae (19) to (22),w4a=1−(cα+t)/C  (19) w4b=(cα+t)/C  (20) w5a=(cα+t)/C  (21) w5b=1−(cα+t)/C  (22) wherein t=(|f2−fin|)/{(|f1−fin|)(|f2−fin|)}, α=fin/f0, fin represents an original image signal, f0 represents a reference value, c represents a constant, and C represents a constant.

26. An image processing method as defined in claim 24 in which the weighting factors w4a, w4b, w5a and w5b are defined by the following formulae (23) to (26),w4a={c(1−α)+t)}/C  (23) w4b=1−{c(1−α)+t)}/C  (24) w5a=1−(cα+t)/C  (25) w5b=(cα+t)/C  (26) wherein t=(|f2−fin|)/{(|f1−fin|)(|f2−fin|)}, α=fin/f0, fin represents an original image signal, f0 represents a reference value, c represents a constant, and C represents a constant.

27. An image processing system as defined in claim 15 in which the processing for suppressing granularity by the granularity suppressing means involves granularity suppressing processing using morphology operation and the processing for enhancing sharpness by the sharpness enhancing means involves processing in which the original image signal is separated into a low frequency component, an intermediate frequency component and a high frequency component, the high frequency component is enhanced while the intermediate frequency component is suppressed, and the processed high frequency component, the processed intermediate frequency component and the low frequency component are combined into an image signal.

28. An image processing system as defined in claim 27 in which the granularity suppressing processing using morphology operation is defined by the following formula (27), f1={fopn⁢ ⁢(when⁢ ⁢fcls=fin)fcls⁢ ⁢(when⁢ ⁢fopn=fin)(fopn+fcls)/2⁢ ⁢(when⁢ ⁢fcls≠fin, ⁢and⁢ ⁢fopn≠fin)(27)wherein f1 represents the granularity-suppressed image signal, fcls represents a closing image signal, fopn represents an opening image signal, and fin represents the original image signal.