Patent Application
20140285682
1. Field of the Invention
The present technique relates to an image processing apparatus, an image processing method, and an imaging apparatus which are capable of providing various effects by performing color conversion processing on image data.
2. Description of the Related Art
Imaging apparatuses use image processing apparatuses configured to perform image processing in accordance with user preference on captured image. For example, there has been known an image processing apparatus configured to perform color conversion processing (hereinafter, also referred to as one-point color processing) for achromatizing colors other than a color in a user-specified-region (a color to be remained) on a through image (a live-view image) displayed on a display unit equipped with a touch panel function (for example, see Unexamined Japanese Patent Publication No. 2012-244337).
The imaging apparatus according to the present technique has a face detection unit configured to detect a face region which is determined as a face in an image, a position specifying unit configured to specify a particular position in the image in response to user operation, a range specifying unit configured to determine a first range as a range for calculating an evaluation value which is a central value of a hue and to set a hue range at a position specified by the position specifying unit in a case where the position specified by the position specifying unit is not contained in the face region, and to determine a second range as a range for calculating an evaluation value which is the central value of a hue and to set a hue range at a position specified by the position specifying unit in a case where the position specified by the position specifying unit is contained in the face region, the second range being smaller than the first range, and an image processing unit configured to perform image conversion processing based on color information contained in the hue range set by the range specifying unit.
Embodiments will be described below with reference to the drawings about an image processing apparatus, an image processing method and an imaging apparatus according to the present technique. However, unnecessarily detailed description may be omitted. For example, detailed description of already known matters and repetition of substantially the same configuration may be omitted. Such omissions are for avoiding unnecessary redundancy in the following description to facilitate understanding by those skilled in the art.
The applicant provides the accompanying drawings and the following description for those skilled in the art to fully understand the present technique and does not intend to limit the subject described in the claims by the accompanying drawings and the following description.
A lens barrel accommodating optical system 1 and flash 2 are arranged on a front of a housing of the digital camera. On a top of the housing, an operation unit including still image release button 3, zoom lever 4, and power button 5 is arranged. On a back of the housing, liquid crystal monitor 6 which is a display unit equipped with a touch panel function for displaying a through image or a photographed image of a subject is arranged. Further, an operation unit including center button 7, directional button 8, moving image release button 9, and mode switch 10 is arranged on the back.
As illustrated in
CCD image sensor 20 captures the subject image formed through optical system 1 and generates the image data. When the digital camera is in a photographing mode, CCD image sensor 20 can generate a new frame of image data for every certain time period.
In AFE 21, noise suppression by correlated double sampling, amplification to an input range width of an A/D converter by an analog gain controller unit, and A/D conversion by the A/D convertor are performed on the image data read out from CCD image sensor 20. Subsequently, AFE 21 outputs the image data to image processing unit 22.
Image processing unit 22 performs various types of image processing on the image data output from AFE 21. The various types of image processing include smear correction, white balance correction, gamma correction, YC conversion processing, electronic zoom processing, compression processing, and expansion processing. The image processing is not limited to those types of processing. Image processing unit 22 temporarily stores the image information subjected to the various types of image processing in buffer memory 23. Image processing unit 22 performs color conversion processing including one-point color processing on the image data stored in buffer memory 23. The image data after the color conversion processing is stored in buffer memory 23. Details of the one-point color processing will be described later. Meanwhile, buffer memory 23 is a storing means functioning as a work memory for image processing unit 22 or for controller unit 24 and can be implemented by a DRAM (Dynamic Random Access Memory) or the like.
Controller unit 24 is for controlling the operation of the whole digital camera and is implemented by a microcomputer which has a ROM (Read Only Memory) for storing programs, a CPU (Central Processing Unit) for processing various types of information by executing the programs, and the like. The ROM stores programs for collectively controlling the operation of the whole digital camera including file control, autofocus control (AF control), automatic exposure control (AE control), and emission control for flash 2. Controller unit 24 serves as face detection unit 24A, position specifying unit 24B, and range specifying unit 24C by executing the programs stored in the ROM.
Face detection unit 24A detects a region judged to be a person's face from the image data stored in buffer memory 23 to take the region as a face region and stores information of the face region in buffer memory 23. Various publicly known techniques can be used as a method for detecting a region which is a person's face from the image data.
Position specifying unit 24B pinpoints a position or a region in the image based on user operation and stores it in buffer memory 23 as the position or the region of a “color desired to be remained”.
Range specifying unit 24C determines a range for calculating an evaluation value and a width of the hue based on the position or the region of a “color desired to be remained” pinpointed by position specifying unit 24B and the face detection result from face detection unit 24A. Then, range specifying unit 24C determines the range of the color desired to be remained as a hue range based on the range for calculating an evaluation value and the width of the hue. The range for calculating an evaluation value and the width of the hue will be described in detail later.
Memory card 25 for recording the image data can be attached to/detached from card slot 26 connected to controller unit 24 by the user so that it is electrically and mechanically connected to controller unit 24. Card slot 26 may be provided with a function of controlling memory card 25. Memory card 25 is an external memory that contains a recording unit such as a flash memory. Memory card 25 records data such as image data to be processed in image processing unit 22. The high pixel count image data to be recorded on memory card 25 is generated by image processing unit 22 based on the image data generated by CCD image sensor 20 after detection of user operation on the release button. Controller unit 24 records on memory card 25 the image data processed in image processing unit 22 and stored in buffer memory 23. The image data recorded on memory card 25 is displayed on liquid crystal monitor 6 based on user operation.
Flash memory 27 connected to controller unit 24 functions as an internal memory for recording the image data and setup information of the digital camera. Although flash memory 27 is illustrated to be external to controller unit 24 in
Optical system 1 includes focus lens 11, zoom lens 12, diaphragm 13, and shutter 14. Focus lens 11 is used for adjusting a focusing state of the subject. Zoom lens 12 is used for adjusting an angle of view of the subject. Diaphragm 13 is used for adjusting a light quantity incident on CCD image sensor 20. Shutter 14 adjusts an exposure time of light incident on CCD image sensor 20. Focus lens 11, zoom lens 12, diaphragm 13, and shutter 14 are driven by respectively corresponding driving means such as a DC motor and a stepping motor according to control signals sent from controller unit 24. Although not illustrated in the drawings, optical system 1 may include an Optical Image Stabilizer (OIS) lens. The respective types of lens included in optical system 1 can be implemented by a plurality of lenses or a plurality of lens groups.
Liquid crystal monitor 6 displays an image based on the image data processed by image processing unit 22. Liquid crystal monitor 6 displays the images such as the through image and the recorded image. The through image is an image successively displaying a new frame of image data which is generated by CCD image sensor 20 for each certain time period. In general, when the digital camera is in the photographing mode, image processing unit 22 generates the through image based on the image data generated by CCD image sensor 20. With the through image displayed on liquid crystal monitor 6 for reference, the user can take a photograph while confirming the composition of the subject. The recorded image is made as a result of reducing the high pixel count image data recorded on memory card 25 to low pixel count image data during a playback mode of the digital camera.
Operation unit 28 connected to controller unit 24 is for detecting user operation and collectively refers to operation buttons and switch of the digital camera described with reference to
Still image release button 3 is a two-stage push button which allows a half-press operation and a full-press operation. When still image release button 3 is half-pressed by the user, controller unit 24 performs AF (Auto Focus) control or AE (Auto Exposure) control to determine photographing conditions. Subsequently, when still image release button 3 is full-pressed by the user, controller unit 24 performs photographing processing. Controller unit 24 records the image data captured at a moment of the full-press operation onto memory card 25 or the like as a still image. Unless otherwise stated, the expression “to press still image release button 3” is assumed to be corresponding to “to full-press”.
Moving image release button 9 is a push button for instructing start/finish of moving image recording. When moving image release button 9 is pressed by the user, controller unit 24 records the image data generated by image processing unit 22 based on the image data generated by CCD image sensor 20 onto a recording medium such as memory card 25 successively as a moving image. When moving image release button 9 is pressed again, the recording of the moving image finishes.
Zoom lever 4 is for adjusting the angle of view between a wide-angle end and a telephoto end inclusive. Zoom lever 4 is implemented by a lever which automatically returns to a middle position. When zoom lever 4 is operated by the user, it sends an operation instructing signal for driving zoom lens 12 to controller unit 24. In other words, when zoom lever 4 is set to the wide-angle end, controller unit 24 drives zoom lens 12 to capture a wide-angle shot of the subject. Conversely, when zoom lever 4 is set to the telephoto end, controller unit 24 drives zoom lens 12 to capture a telephoto shot of the subject. Power button 5 is a push button for the user to instruct the digital camera to supply power to the respective components of the digital camera. When power button 5 is pressed by the user in the power OFF state, controller unit 24 supplies power to the respective components of the digital camera to activate them. When power button 5 is pressed by the user in the power ON state, controller unit 24 operates to stop supplying power to the respective components.
Center button 7 is a push button. When center button 7 is pressed by the user while the digital camera is in the photographing mode or the playback mode, controller unit 24 displays a menu screen on liquid crystal monitor 6. The menu screen is for the user to set respective conditions for photography operation/playback operation. The information set on the menu screen is recorded on flash memory 27. In the case where the respective conditional setting items have been selected and center button 7 is pressed, center button 7 functions as a button for deciding on the set conditions.
Directional button 8 is a push button which can be pressed in upward/downward/leftward/rightward directions. By pressing directional button 8 in any one of the directions, the user can select one of the respective conditional items displayed on liquid crystal monitor 6.
Mode switch 10 is a button which can be pressed in upward/downward directions. By pressing mode switch 10 in either direction, the user can switch the state of the digital camera to the photographing mode or the playback mode.
Image processing unit 22 and controller unit 24 may be implemented by hardwired electronic circuits or may be implemented by a microcomputer or the like which is operated according to programs. Alternatively, image processing unit 22 and controller unit 24 may be formed together on a single semiconductor chip. The ROM needs not to be provided inside controller unit 24 and may be provided outside of controller unit 24 as an external memory.
An imaging device constituting the imaging unit is not limited to the CCD image sensor and may be another sensor such as a CMOS (Complementary Metal-Oxide Semiconductor) image sensor. Further, an imaging unit formed of an optical system, an image sensor, and the like may be implemented by a camera module which has lenses, a CMOS image sensor, part of signal processing circuit unit, and the like integrated into a module.
Operation of the digital camera including operation of the image processing apparatus according to the embodiment of the present technique will be described below.
The image processing apparatus in the digital camera of the embodiment can perform the one-point color processing on captured image data. The digital camera displays the image data, and based on the user-specified position of a color desired to be remained and the face detection result, determines a range for calculating an evaluation value and a hue width. Then, the digital camera can determine the color to be remained based on the determined range for calculating an evaluation value and the determined hue width and perform color conversion processing for achromatizing colors other than the color to be remained.
[2-1. Photography Operation of the Digital Camera]
First, the photography operation of the digital camera will be described.
When the digital camera enters the photographing mode in response to user operation performed on mode switch 10, controller unit 24 performs initialization step S401 for initialization processing necessary to record a still image.
Next, in mode judgment step S402, controller unit 24 checks the operated state of mode switch 10, and in the case where it judges that the switch is set to the photographing mode (Y), it performs display processing of display step S403 for displaying a through image on liquid crystal monitor 6 according to current set values. Then, in release button operation judgment step S404, controller unit 24 checks the operated state of still image release button 3, and in the case where it judges that still image release button 3 is pressed and it is commanded to perform the photography operation (Y), it performs processing of photographing step S405 for photographing a still image. Specifically, controller unit 24 records image data of a still image which has been generated by image processing unit 22 based on image data generated by CCD image sensor 20 onto a recording medium such as memory card 25 at the moment when still image release button 3 is pressed.
After performing the processing of still image photographing operation of photographing step S405, controller unit 24 repeatedly performs the processing from mode judgment step S402.
On the other hand, in the case where controller unit 24 judges that the state of mode switch 10 is not the photographing mode (N) in mode judgment step S402, it finishes the processing without performing operation of next display step S403. In the case where controller unit 24 has not detected depression of still image release button 3, i.e., in the case where controller unit 24 judges that it is not commanded to perform the photography operation (N) in release button operation judgment step S404, it repeatedly performs processing from mode judgment step S402.
[2-2. Operation of the One-Point Color Processing]
Now, operation of the one-point color processing in the image processing apparatus of the digital camera according to the present technique will be described.
As illustrated in
In the case where range specifying unit 24C judges that position 6b specified by position specifying unit 24B is not contained in face region 6a, it decides a default standard first range to be a range for calculating an evaluation value which is a central value of a hue as illustrated in
On the other hand, in the case where range specifying unit 24C judges that position 6c specified by position specifying unit 24B is contained in face region 6a, it decides a second range to be the range for calculating an evaluation value which is a central value of a hue, the second range being smaller than the first range as illustrated in
As a result, in the case where position 6b specified by position specifying unit 24B is not contained in face region 6a, hue H1 indicated by an evaluation value calculated on the basis of the pixels in the first range (width X0×height Y0) is decided to be a center and the hue range (H1±Δh) with Δh indicating the hue width added to the center is set to the hue range of a color desired to be remained as illustrated in
Although the first range (width X0×height Y0) is described as the default range in the description with reference to
Further, the same hue width Δh needs not to be added to both sides of hues H1 and H2 present at the positions specified by position specifying unit 24B to form the hue ranges respectively, and the hue ranges only need to contain hues H1 and H2 present at the specified positions respectively. For example, in
Then, image processing unit 22 performs image conversion processing based on color information contained in the hue range (H1±Δh) or in the hue range (H2±Δh) specified by range specifying unit 24C.
As described above, in the image processing apparatus of the digital camera according to the present technique, the second range with the position of a color to be remained set at the center in the case where the position of a color desired to be remained is contained in the face region is specified smaller than the first range with the position of a color to be remained set at the center in the case where the position of a color desired to be remained is not contained in the face region. As a result, in the case where the user has specified the face of a person with the intention of making the color of the face of the person remained, the above described configuration can enhance the possibility of making the whole of the smaller range with a less number of pixels be contained in the face of the person. Consequently, the configuration decreases a risk of having pixels other than the skin color mixed with the pixels in the second range when the mean value of the hues of pixels in the second range is acquired as the evaluation value, therefore, one-point color processing much nearer to the user's intention becomes available.
As illustrated in
In image selection step S502, controller unit 24 prompts the user to select image data to be subjected to the one-point color processing. Specifically, controller unit 24 causes liquid crystal monitor 6 to display a list of image data recorded on memory card 25. Then, according to an operation performed by the user on operation unit 28, controller unit 24 selects a piece of image data from the list.
Next, in face detection step S503, face detection unit 24A detects a face region in the selected image data. In the case where the image data contains a region which can be judged to be a face of a person, the region is stored in buffer memory 23 as a set of a center position, a length and a width of the region. Then, controller unit 24 performs display step S504 for displaying an image based on the image data on liquid crystal monitor 6. On that occasion, controller unit 24 causes a frame enclosing face region 6a to be superimposed on the image displayed on liquid crystal monitor 6 as illustrated in
Subsequently, position specifying unit 24B performs position specifying step S505 for prompting the user to select a position in the image for the user to decide on a color desired to be remained. Specifically, when the user operates directional buttons 8 or the like while liquid crystal monitor 6 is displaying the image, position specifying unit 24B moves a cursor on the image based on the user operation. In the case where the user presses center button 7 when the cursor has moved to a desired position, position specifying unit 24B stores the position of the cursor at the moment in buffer memory 23 as the position of a color desired to be remained. As another example, the image processing apparatus may be configured to have a touch panel function added to liquid crystal monitor 6 so that the user selects the position by directly touching the screen of liquid crystal monitor 6.
Next, range specifying unit 24C performs face region judgment step S506 for judging whether the position of a color desired to be remained acquired in position specifying step S505 is in the face region detected in face detection step S503. In the case where range specifying unit 24C judges that the position of a color desired to be remained acquired in position specifying step S505 is in the face region (Y), it sets the range for calculating the evaluation value to the second range smaller than a default value (standard) first range as described above in range specifying step S507.
Further, in range specifying step S508, range specifying unit 24C sets the hue range bigger than the default range. Here, it is assumed that H2 is the hue of a color desired to be remained and ΔH0 is the standard hue width. Specifically, in the case where range specifying unit 24C judges that the position of a color desired to be remained selected in position specifying step S505 is in the face region (Y), it sets the hue width ΔH1 which satisfies a condition ΔH1>ΔH0. Therefore, the hue range results in the range H2±ΔH1.
On the other hand, in the case where range specifying unit 24C judges that the position of a color desired to be remained acquired in position specifying step S505 is not in the face region (N), it sets the range for calculating the evaluation value to the default value (standard) first range as described above in range specifying step S509.
Further, in range specifying step S510, range specifying unit 24C sets the hue range of the color desired to be remained to the default range. Here, it is assumed that H1 is the hue of a color desired to be remained and ΔH0 is the standard hue width. Specifically, in the case where range specifying unit 24C judges that the position of a color desired to be remained selected in position specifying step S505 is not in the face region (N), it sets the hue width ΔH0 and the hue range becomes the range H1±ΔH0.
Now, the range for calculating the evaluation value will be described. In finally determining the hue range of the color desired to be remained, the hue which would come to the center (median value) of the hue range is determined as the evaluation value. The evaluation value can be calculated as a mean value of hues of surrounding pixels in a particular region around the user-specified position. The particular region on that occasion is the range for calculating the evaluation value. Specifically, in the case where the range for calculating the evaluation value is rectangular, the range for calculating the evaluation value is set to satisfy expressions X1<X0 and Y1<Y0, where X0 is the width and Y0 is the height of a default standard range for calculating the evaluation value and X1 is the width and Y1 is the height of the range in the case where it is judged that the position of a color desired to be remained is in the face region. The hue range indicates a color to be remained in the form of a hue range of the color as a range from the evaluation value around the hue indicated by the evaluation value.
Next, range specifying unit 24C calculates the evaluation value which is the central value of the hue in evaluation value calculation step S511. Specifically, range specifying unit 24C calculates a mean value of hues of pixels in a range for calculating the evaluation value which has been set in range specifying step S507 or S509 around the position of a color desired to be remained selected in position specifying step S505, and takes the mean value as the evaluation value.
Next, range specifying unit 24C determines the hue range as described above in hue range setting step S512.
Next, controller unit 24 performs the color conversion processing in image processing step S513. Specifically, controller unit 24 performs the color conversion processing by controlling image processing unit 22. Image processing unit 22 takes color information contained in the hue range at the position specified by position specifying unit 24B (the hue range determined in hue range setting step S512) as standard color information and performs the image conversion processing for changing color information of a particular region of the image wherein the particular region of the image has color information which is not contained in the standard color information. In the case of the one-point color processing, image processing unit 22 performs the image conversion processing for achromatizing a particular region of the image wherein the particular region of the image has color information which is not contained in the standard color information.
The color conversion processing is not performed on pixels which have hues within the hue range. Then, image processing unit 22 performs display step S514 for displaying the image which has been subjected to the color conversion processing in image processing step S513 on liquid crystal monitor 6. Finally, image processing unit 22 performs record step S515 for recording the image data which has been subjected to the one-point color processing on flash memory 27.
In the above embodiment, although a case of one-point color processing for achromatizing a particular region in the image has been described as an example of image conversion for changing color information of a particular region of the image wherein the particular region of the image has color information which is not contained in the standard color information, the image conversion may be processing for converting the color information into a chromatic color such as sepia. Further, image conversion for changing brightness or contrast of an image may be performed on the position specified by position specifying unit 24B. Moreover, in converting the color information of a particular region which has color information different from the standard color information, the processing for converting the color information may be performed also on the position specified by position specifying unit 24B on the condition that the color information is converted into color information other than the color information of the particular region.
The digital camera which has the image processing apparatus according to the present technique has an imaging unit for capturing a subject image, a data recording unit for recording image data generated on the basis of the subject image captured by the imaging unit, a display unit for displaying an image based on the image data recorded in the data recording unit, and an image processing apparatus configured to perform image processing on the image displayed on the display unit.
The image processing apparatus has controller unit 24 configured to determine a first range as a range for calculating an evaluation value which is a central value of a hue and to set a hue range at a position specified by user operation in a case where a particular position in the image specified by user operation is not contained in the face region of the image, and to determine a second range as a range for calculating an evaluation value which is a central value of a hue and to set a hue range at the specified position in a case where a particular position in the image specified by user operation contains the face region, the second range being smaller than the first range.
Controller unit 24 has face detection unit 24A configured to detect a face region which is determined as a face in an image, position specifying unit 24B configured to specify a particular position in the image in response to user operation, and range specifying unit 24C configured to determine a first range as a range for calculating an evaluation value which is a central value of a hue and to set a hue range at a position specified by position specifying 24B in a case where the position specified by position specified unit 24 B is not contained in the face region, and to determine a second range as a range for calculating an evaluation value which is a central value of a hue and to set a hue range at a position specified by position specifying 24B in a case where the position specified by position specifying 24B contains the face region, the second range being smaller than the first range. Image processing unit 22 of the image processing apparatus performs image conversion processing based on color information contained in the hue range set by range specifying unit 24C of controller unit 24.
With the above described configuration, the second range with the position of a color to be remained set at the center in the case where the position of a color desired to be remained is contained in the face region is specified smaller than the first range with the position of a color to be remained set at the center in the case where the position of a color desired to be remained is not contained in the face region. As a result, in the case where the user has specified the face of a person with the intention of making the color of the face of the person remained, the above described configuration can enhance the possibility of making the whole of the smaller range with a less number of pixels be contained in the face of the person. Consequently, the configuration decreases a risk of having pixels other than the skin color mixed with the pixels in the second range when the mean value of the hues of pixels in the second range is acquired as the evaluation value, therefore, one-point color processing much nearer to the user's intention becomes available.
The present technique is not limited to the above described embodiment. Other embodiments of the present technique will be described below.
In the embodiment described in
Further, in the embodiment described in
The embodiment has been provided by the accompanying drawings and the detailed description as described above. The present inventor provides the accompanying drawings and the detailed description to exemplify the subject matter defined in the claims with reference to a particular embodiment for those skilled in the art. Therefore, the constituent elements illustrated in the accompanying drawings and described in the detailed description may include not only a constituent element which is necessary to solve the problem but also a constituent element which is not necessary to solve the problem. Accordingly, it should not be instantly understood that the unnecessary constituent element is necessary only because the unnecessary constituent element is illustrated in the accompanying drawings or discussed in the detailed description. Further, various changes, substitutions, addition and/or omission and the like may be performed on the embodiment within the scope of the claims and the equivalent of the claims.
Since color conversion processing nearer to the user's intention becomes available according to the present technique, the technique can be also applied to a digital still camera, a digital video camera, a mobile phone, a smart phone, a mobile PC, and the like.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2013-056345 | Mar 2013 | JP | national |
| 2014-020934 | Feb 2014 | JP | national |
