The present application claims priority from Japanese Application P2004-187291 filed on Jun. 25, 2004, the content of which is hereby incorporated by reference into this application.
1. Field of the Invention
The present invention relates to a technique of processing input color image data representing a color image.
2. Description of the Related Art
The recent rapid spread of digital still cameras and other diverse imaging devices ensures easy accessibility of images in the form of digital data. Color image data consists of three color component data of R, G, and B images corresponding to three primary colors of light. The digital still cameras and other imaging devices output RGB tone value data as color image data. Display monitors and printers receive these RGB tone value data and display and print color images based on the RGB tone value data.
Some specific colors, that is, flesh color, green, and sky blue generally have emphasized visual effects in a color image. The use easily identifies, for example, even a slightly reddish tint or a slightly dim tint on the flesh color in the color image, as well as even slight differences among greens and sky blues in the color image. The flesh color, green, and sky blue are memory colors that are visually recognizable in a distinctive manner from other colors. An intuitive correction technique is thus preferable for such memory colors in the color image.
An HSI expression format using hue H, saturation S, and intensity I naturally allows the hue H, the saturation S, and the intensity I of each color image to be more readily processed than a conventional RGB expression format. This HSI expression format is widely applied to the intuitive correction procedure. One proposed technique converts RGB image data representing a color image into HSI data in the HSI expression format and subsequently corrects the hue H, the saturation S, and the intensity I of the color image (for example, U.S. Pat. No. 3,194,341).
This prior art technique sequentially corrects the hue H, the saturation S, and the intensity I of the whole converted HSI data. This undesirably increases the processing load of correction and extends the total data processing time. This sequential correction of the hue H, the saturation S, and the intensity I may cause the results of the correction of the hue H to be affected by subsequent correction of the saturation S and the intensity I. This may cause the user to feel some oddness and strangeness in the picture quality of a resulting corrected color image.
The object of the invention is thus to eliminate the drawbacks of the prior art techniques and to make intuitively comprehensible image correction of RGB color image data after conversion to HSI data in an intuitive HSI expression format, while ensuring no substantial change of the picture quality of a resulting color image by correction.
In order to attain at least part of the above and the other related objects, an image processing device of the invention inputs color image data representing a color image, and sets a color correction area as an object correction range and a degree of correction according to a correction rate curve in the color correction area with regard to each of multiple specific colors, for example, multiple memory colors of flesh color, green, and sky blue. The image processing device sets the color correction areas and the correction rate curves, based on expression of the color image data in respective hue ranges of the multiple specific colors.
In one preferable application, the image processing device converts sampling data extracted from the color image data into HSI data in an HSI expression format using hue H, saturation S, and intensity I, specifies expression of the converted HSI data, and sets the color correction areas and the correction rate curves based on the specified expression of the HSI data. This arrangement desirably decreases the processing data volume for specification of the color correction areas and the correction rate curves, thus effectively reliving the operation load.
The image processing device of the invention converts the whole input color image data in the specific format into HSI data in the HSI expression format. The image processing device then corrects the hue H, the saturation S, and the intensity I of the HSI data in the HSI expression format with correction rate curves set in respective color correction areas of the hue H, the saturation S, and the intensity I, with regard to each of the multiple specific colors. Subsequently, the image processing device reversely converts the corrected HSI data into color image data in the specific format. The reversely converted color image data are output to an output device, for example, a monitor display or a printer.
The technique of the invention converts color image data in the specific format into HSI data in the intuitive HSI expression format and subsequently corrects the converted HSI data. This conversion to the HSI data enables the user to intuitively follow the correction. Only the hue ranges of the multiple specific colors are the target of correction of the HSI data with regard to the hue H, the saturation, and the intensity I. Namely correction is not made over the whole hue range. This simple procedure desirably relives the load of the correction operation.
The multiple specific colors, flesh color, green, and sky blue, do not have any overlaps in the hue H of the HSI data in the HSI expression format. Correction with a correction rate curve set for the hue range of one specific color affects only the HSI data in a preset color correction area in the hue range of the specific color, while not affecting correction in other color correction areas in the hue ranges of the other specific colors. The prior art technique sequentially corrects the whole HSI data with regard to the hue H, the saturation S, and the intensity I. The results of the correction of the hue H may thus be affected by subsequent correction of the saturation S and the intensity I. The technique of the invention, however, does not need sequential correction of the whole HSI data with regard to the hue H, the saturation S, and the intensity I, thus ensuring no substantial change of the picture quality of a resulting color image by correction. The prior-art sequential correction technique requires multiple cycles of correction operation, while the technique of this invention requires only one cycle of correction operation. This desirably relieves the operation load.
One preferable embodiment of the image processing device of the invention corrects each of the hue H, the saturation S, and the intensity I of the HSI data in the HSI expression format with a corresponding correction rate curve, which sets a fixed correction rate in a substantial color correction zone in a center of a corresponding color correction area and gives smaller correction rates than the fixed correction rate in a pre-zone and in a post-zone before and after the substantial color correction zone in the color correction area. This arrangement desirably avoids any abrupt changes of the corrected HSI data across the boundaries of the respective color correction areas, thus effectively preventing the user from feeling some oddness and strangeness in the picture quality of a resulting corrected color image.
Another preferable embodiment of the image processing device of the invention specifies a variation of each correction rate curve in each color correction area as the object correction range to set a correction rate of 0 to a specific point of a hue value having best match with each of the multiple specific colors. The procedure of this embodiment does not make any correction at specific points of hue values having the best matches with the respective specific colors, flesh color, green, and sky blue, while making the correction for surrounding hue values around the respective specific points. This arrangement effectively prevents the user from feeling some oddness and strangeness in the picture quality of a resulting corrected color image with regard to these specific colors.
The image data processing technique of the invention is actualized by diversity of applications other than the image processing device, for example, a corresponding image data processing method, computer programs that cause a computer to attain the functions of the image processing device or the corresponding image data processing method, and recording media that store such computer programs.
In order to clarify the features, aspects, and effects of the invention, one mode of carrying out the invention is described below as a preferred embodiment in the following sequence:
A1. Configuration of Image Processing System
The image database 20 includes various imaging devices, such as a digital video camera 21 and a digital still camera 22, and diverse image data storage units, such as a DVD 23, a hard disk 24, and a memory card 25, and supplies image data to the personal computer 30. The DVD 23, the hard disk 24, and the memory card 25 store color image data representing color still images taken by the imaging device, for example, the digital still camera 22. The color image data are constructed as R, G, and B tone value data of R, G, and B images corresponding to three primary colors of light.
The personal computer 30 is designed to output edited color images as RGB tone value data to the color printer 50 and a display 43 of the user interfaces 40, as discussed later.
The personal computer 30 includes a CPU, a ROM, a RAM, and a hard disk with image processing software installed therein, although these constituents are not specifically illustrated. These constituents cooperatively exert required functions for image processing, that is, a correction setting module, an expression format conversion module, a correction execution module and a reverse conversion module. The personal computer 30 receives and sends data from and to the external devices, such as the image data input unit 28, the display 43, and the color printer 50, via non-illustrated I/F circuits. The image processing software installed in the hard disk is executed to correct color image data input by the image data input unit 28. The details of this image processing flow (image editing process) will be described below. The image data input unit 28 may be incorporated in the personal computer 30.
A2. Image Editing Process
The image editing process (image data processing) of
In the image editing process, the personal computer 30 first inputs color image data of each color image as a possible option of an editing object from the image database 20, for example, the digital still camera 22 or the memory card 25, via the image data input unit 28 and shows a color image expressed by the input color image data (RGB data) on the display 43 (step S200). The color image data are RGB data corresponding to three primary colors and have RGB tone values of individual pixels.
When simultaneous input of image data representing multiple images is allowed, a list of the simultaneously input images may be displayed as thumbnail images, for example, in a right-half display area on the display 43. The display of the simultaneously input multiple images may otherwise be sequentially switched on the display 43. The user selects an object image to be edited among the displayed multiple images through the keyboard operation and the mouse operation. The personal computer 30 waits for the user's selection of the object image. In the case of input of image data representing only one image, however, the personal computer 30 does not wait for the user's operation but immediately displays the input image on the display 43 at step S200.
The user may select object image data out of a list of the names of image data, instead of the displayed images.
The personal computer 30 conducts sampling and extraction from the input color image data according to a predetermined rule (step S210). The sampling technique is not specifically restricted but may be adequately selected according to the operation load of image analysis (discussed later) and other relevant factors. For example, the sampling process may perform sampling and extraction from color image data in a (m×n) matrix to a data volume of 1/100 to 1/1000 of the total data volume. Typical sampling techniques applicable here include systematic sampling, random sampling, and importance sampling.
The personal computer 30 converts the extracted sampling RGB image data into HSI sampling data expressed in an HSI format called an HSI hex-cone color model (step S220). The hex-cone color model is used to convert image data expressed by R, G, and B tone values into another expression format of the hue H, the saturation S, and the intensity I. The HSI hex-cone color model is widely applied as a technique of simply and intuitively manipulating an image in terms of the hue H, the saturation S, and the intensity I. The outline of data conversion by the HSI hex-cone color model is described below.
The RGB image data is tone data and is expressible as a point in a cube (color solid) of 255 on each side when the available tone value range is 0 to 255. For example, image data of black (K) has R, G, and B tone values all equal to 0 and is thus expressible as an apex having coordinates (0,0,0). Image data of white (W) has R, G, and B tone values all equal to 255 and is thus expressible as an apex having coordinates (255,255,255). Similarly red (R), green (G), and blue (B) are respectively expressible as apexes having coordinates (255,0,0), (0,255,0), and (0,0,255). Cyan (C) complementary to red (R) is expressible as an apex of coordinates (0,255,255) that faces the apex of red (R). Magenta (M) complementary to green (G) is expressible as an apex of coordinates (255,0,255) that faces the apex of green (G). Yellow (Y) complementary to blue (B) is expressible as an apex of coordinates (255,255,0) that faces the apex of blue (B).
The HSI hex-cone color model sets a K-W axis of the color solid to an axis I, projects the respective coordinates of the color solid on a plane perpendicular to the axis I, and computes the saturation S and the hue H of image data on the projected plane. The intensity I is directly computable from the coordinates of the color solid. For example, the intensity I of given image data (R,G,B) is computed according to Equation (1):
I=max(R,G,B) (1)
where max(R,G,B) represents a function of selecting the maximum among the R, G, and B tone data. When the image data (R,G,B) is given as a point P in the color solid, the saturation S and the hue H are computed from the coordinates of a projected point P′, which is set by projection of the point P on the plane perpendicular to the axis I as shown in
Projection on the plane perpendicular to the axis I converts the apexes R, Y, G, C, M, and B of the color solid to apexes of a regular hexagon and the apexes K and W of the color solid to a center O of the regular hexagon, as shown in
S=255(1−i)/I (2)
where i=min(R,G,B), which represents a function of selecting the minimum among the R, G, and B tone data, and I denotes the intensity I computed by Equation (1) given above. As clearly shown by Equation (2), the saturation S is indefinite for the intensity I=0.
Projection of the apexes R, Y, G, C, B, and M of the color solid to the apexes R, Y, G, C, B, and M of the regular hexagon shown in
when R=I,
H=255(b−g)/6 (3)
when G=I,
H=255(2+r−b)/6 (4)
when B=I,
H=255(4+g−r)/6 (5)
where r=(I−R)/(I−i)
g=(I−G)/(I−i)
b=(I−B)/(I−i)
When the computed value H<0, 255 is to be added to the computed value H.
The image editing process of
The personal computer 30 then performs image analysis based on the converted sampling HSI data (step S230). The image analysis sets distributions of the sampling HSI data of the analyzed image with regard to specific memory colors, that is, flesh color, green, and sky blue, which are memorable because of the visual characteristics.
Because of the characteristics of the HSI hex-cone color model, the hue H does not have any overlap among hue ranges of the specific memory colors, that is, flesh color, green, and sky blue. A distribution of the sampling HSI data with regard to the hue H is shown in
In the distribution of the sampling HSI data, when the actual hue range of flesh color in the analyzed color image is substantially equal to or mostly overlaps the flesh color hue range Sk/ok, the visual representation of flesh color sufficiently looks flesh color in the color image. In the illustrated distribution of
The saturation S and the intensity I of the specific memory colors, that is, flesh color, green, and sky blue, have some overlaps in the respective hue ranges. The sampling HSI data gives distributions of the saturation S and the intensity I in the respective hue ranges of these specific memory colors. The image analysis thus enables detection of insufficiencies and excesses of the saturation S and the intensity I.
The personal computer 30 sets a color correction area as an object correction range and a correction rate curve in the color correction area with regard to each of the hue H, the saturation S, and the intensity I in each of the hue ranges of the specific memory colors, that is, flesh color, green, and sky blue, based on the results of the image analysis, so as to complete a correction table including such settings (step S240).
At step S240 in the flowchart of
After creation of the correction table as shown in
When it is determined at step S255 that the hue data H of the converted HSI data is within the color correction area of the hue H, on the other hand, the personal computer 30 refers to the correction table of
In the illustrated example of
When the original hue data Horg is within the preset color correction area defined by SkH1 through SkH4, on the other hand, the original hue data Horg is corrected with a preset correction rate curve Δ h to give the corrected hue data H′dh =(1+Δ h) Horg. The correction rate curve Δ h for correction of Horg to H′dh gives a fixed value in a substantial color correction zone between SkH2 and SkH3 and varying values in a pre-zone between SkH1 and SkH2 and in a post-zone between SkH3 and SkH4.
The hue H is corrected with the correction rate curve Δ h set in the correction table of
Subsequently, the personal computer 30 reversely converts the corrected HSI data (having corrected hue H′, corrected saturation S′, and corrected intensity I′) to RGB image data (R′,G′,B′) in the original RGB format (step S270), and determines whether processing has been completed with regard to all the input color image data (step S280). When there is any unprocessed color image data, the processing of steps S250 to S280 is repeated. When all the input color image data have been processed, the personal computer 30 updates the RGB image data input at step S200 to the reversely converted RGB image data (R′,G′,B′) and outputs the updated RGB image data (R′,G′,B′) to the display 43 or to the color printer 50 (step S290). The display 43 shows a resulting color image based on the reversely converted RGB image data (R′,G′,B′), whereas the color printer 50 prints and outputs a resulting color image based on the reversely converted RGB image data (R′,G′,B′).
The reversely converted RGB image data (R′,G′,B′) output at step S290 include the corrected and reversely converted RGB image data (R′,G′,B′) (steps S260 and S270) and the uncorrected and reversely converted RGB image data (R′,G′,B′) (step S275). Since no correction is made, the latter data are identical with the original RGB image data.
As described above, the procedure of the embodiment converts RGB color image data, which represents a color image input from, for example, the digital still camera 22, into HSI data in the intuitive HSI expression format and subsequently corrects the converted HSI data. This conversion to the HSI data enables the user to intuitively follow the correction. Only the hue ranges of flesh color, green, and sky blue are the target of correction of the HSI data with regard to the hue H, the saturation, and the intensity I. Namely correction is not made over the whole hue range. The correction object is the HSI data having the hue H in the preset color correction area (SkH1 to SkH4) of flesh color, the preset color correction areas of green, and the preset color correction area of sky blue shown in
These specific memory colors, flesh color, green, and sky blue, do not have any overlaps in the hue H of the HSI data in the intuitive HSI expression format. Correction with the correction rate curve set for the hue range of flesh color affects only the HSI data in the preset color correction area SkH1 to SkH4 in the hue range of flesh color. As mentioned above, the procedure of this embodiment does not need sequential correction of the whole HSI data with regard to the hue H, the saturation S, and the intensity I. These combined effects ensure no substantial change of the picture quality of a resulting color image by correction.
The color correction table of
In the color correction table of this embodiment shown in
The procedure of the embodiment sets fixed values to correction rates Δ h, Δ s, and Δ i in respective substantial color correction zones of the hue H, the saturation S, and the intensity I in the hue range of flesh color. As shown by the broken lines in
A3. Modified Image Editing Process
The correction table created at step S240 in the flowchart of
In the modified correction table of
The modified correction rate curve keeps the HSI data unchanged at the target point with no correction. The absolute value of the correction rate gradually decreases from the lower limit <2> to the target point and gradually increases from the target point to the upper limit <3> in the substantial color correction zone. Such correction curves are obtained with regard to the hue H, the saturation S, and the intensity I in the respective hue ranges of flesh color, green, and sky blue.
The modified procedure sets the correction rate of the HSI data to 0 at the target point having the best match in visual recognition with flesh color, green, or sky blue. The HSI data at the target point accordingly represents a corresponding target color having the best match in visual recognition with flesh color, green, or sky blue. The correction rate curve gives the smaller value to the hue range closer to the target color. There is accordingly no abrupt change across the target color. The correction rate curve gives the larger value to the hue range apart from the target color. Such correction desirably improves the expression of the target color and expands the expression range of the target color and thus effectively prevents the user from feeling some oddness and strangeness in the picture quality of a resulting color image.
The embodiment and its modification discussed above are to be considered in all aspects as illustrative and not restrictive. There may be many other modifications, changes, and alterations without departing from the scope or spirit of the main characteristics of the present invention.
The procedure of the above embodiment converts the sampling RGB image data into HSI data in the intuitive HSI expression format and subsequently sets color correction areas and correction rate curves for the HSI data. One modified procedure may set color correction areas and correction rate curves for the sampling RGB image data without conversion. The object color space for correction is not limited to the HSI color space, but may be any color space equivalent to the HSI color space, for example, HSB, HSV, and L*a*b* color spaces.
Having described a preferred embodiment of the invention with reference to the accompanying drawings, it is to be understood that the invention is not limited to the embodiments, and that various changes and modifications could be effected therein by one skilled in the art without departing from the spirit or scope of the invention as defined in the appended claims.
Number | Date | Country | Kind |
---|---|---|---|
2004-187291 | Jun 2004 | JP | national |