Patent Application
20210112179
An aspect of the present invention relates to an image processing apparatus performing a correction for deforming an image.
Techniques are known in which an impression of an imaging object is changed by performing image processing to enlarge or reduce a part of an image. For example, these techniques include a technique for correcting the face of a person in the image so as to make the face smaller to provide an impression of a small face, a technique for correcting the whole body of a person in the image so as to make the whole body thinner to provide an impression of a slender body, and a technique for correcting the legs of a person in the image so as to make the legs longer to provide an impression of a good figure. As the technique for correcting a face in an image so as to make the face smaller, included in the above-described techniques, PTL 1 describes a technique in which the face is corrected so as to be smaller by configuring, as a deformation area, an area including a face area of a person and reducing the deformation area. In the technique disclosed in PTL 1, in a case that the configured deformation area is configured outside the image, the correction configured for the outside of the image is omitted to suppress degradation of image quality.
PTL 1: JP 2009-104672 A
However, in the technique described in PTL 1, the correction may or may not be performed depending on the position of the face of the person in the image. Thus, for example, in a case that a user takes an image by using a camera equipped with a function to correct the face in the image so as to make the face smaller, the effect intended by the user may fail to be obtained. For example, in a case that one image depicts multiple persons, a correction may be performed on the faces of persons located in the center of the image, and no correction may be made to the faces of persons located at ends of the image. Then, the impression provided by the image may be such that only some of the persons have a small face, whereas the faces of the persons subjected to no image correction are relatively large.
In view of the foregoing, a main object of an aspect of the present invention is to provide an image processing apparatus, an imaging apparatus, an image printing apparatus, a method for controlling an image processing apparatus, and an image processing program which are capable of more appropriately correcting an image.
An image processing apparatus according to an aspect of the present invention includes a correction processing unit configured to reduce a first area in an image and to enlarge a second area outside the first area to correct the image, wherein in a case that a reduction direction of the first area extends from a first side of the image toward an inside of the first area, the correction processing unit determines whether to clip the first side of the image depending on a distance from the first area to the first side.
An image processing apparatus according to another aspect of the present invention includes a correction processing unit configured to reduce a first area in an image and to enlarge a second area outside the first area to correct the image, wherein in a case that a reduction direction of the first area extends from a first side of the image toward an inside of the first area, the correction processing unit determines whether to change the reduction direction of the first area to prevent the reduction direction from extending from the first side of the image toward the inside of the first area depending on a distance from the first area to the first side.
According to an aspect of the invention, the image can be more appropriately corrected.
An embodiment (Embodiment 1) of the present invention will be described below, based on
First, an example of a configuration of an image printing apparatus 1 according to an embodiment of the present invention will be described, based on
The imaging unit 10 images an imaging object, and transmits a captured image (image including a target area including the imaging object) to the controller 40 as an input image. The target area is an area on which the controller 40 performs reduction and enlargement processing.
The operation unit 20 receives a user input, and is implemented by, for example, a touch panel and a mouse. For example, in a case that the operation unit 20 is a touch panel, the input image is displayed on the display unit 30 including the touch panel.
The display unit 30 displays various images. The display unit 30 displays, for example, the image captured by the imaging unit 10 or an output image generated by an image correction unit 45 described below.
The controller 40 integrally controls the image printing apparatus 1. The controller 40 functions as an image processing apparatus performing image processing on the image (input image) captured by the imaging unit 10, and generates a processed (corrected) output image. A specific configuration of the controller 40 will be described below.
The printing unit 50 prints the output image (image) generated by the processing by the controller 40. The printing unit 50 may further print, on the output image, an image drawn by the user via the operation unit 20.
The storage unit 60 stores, for example, various control programs performed by the controller 40 and includes, for example, a non-volatile storage apparatus such as a hard disk and a flash memory. The storage unit 60 stores the input image and the output image, for example. The storage unit 60 may store parameters and the like necessary for processing by the controller 40, such as image processing (correction processing) and imaging object detection processing.
Note that the image printing apparatus 1 does not necessarily include the controller 40 functioning as an image processing apparatus. For example, an external apparatus capable of being communicatively coupled to the image printing apparatus 1 may include an image processing function of the controller 40.
The image printing apparatus 1 need not include an imaging apparatus including an imaging unit 10 and a controller 40 functioning as an image processing apparatus. In this case, the imaging apparatus may function as an external apparatus that can be communicatively coupled to the image printing apparatus 1 or may not include the function. The image printing apparatus 1 need not include the imaging unit 10, and in this case, for example, the imaging unit 10 functions as an external apparatus that can be communicatively coupled to the image printing apparatus 1.
Now, a specific configuration of the controller 40 will be described, based on
The imaging object detection unit 41 detects an imaging object (target area) to be corrected from the input image input to the controller 40. Examples of the imaging object detected by the imaging object detection unit 41 include (1) a person, (2) the face of the person, (3) an organ of the face such as the eyes, the mouth, or the nose, and (4) a contour of the face. In a case that the imaging object detection unit 41 detects a face, for example, the imaging object detection unit 41 can detect an imaging object using an existing technique, such as utilization of information of a skin color area detected from the input image.
Note that detection of an imaging object by the imaging object detection unit 41 may be manual. In other words, a user may detect an imaging object from an input image. In this case, for example, the imaging object detection unit 41 (imaging object selection unit) causes the display unit 30 to display the input image, and detects (selects) the imaging object to be corrected specified by a user input via the operation unit 20. In a case that multiple imaging objects are present in the input image, the imaging object detection unit 41 selects an imaging object to be corrected, based on the user input.
For example, in a case that the operation unit 20 is a touch panel, touching the touch panel selects the imaging object in the input image to be displayed by the display unit 30. In a case that the operation unit 20 is a mouse, the imaging object is selected, based on a mouse operation.
In the description herein, correction processing is assumed to be performed on an image captured by the imaging unit 10 (i.e., an image including the imaging object). Specifically, in the description, reduction and enlargement processing is assumed to be performed on the imaging object included in the image. However, the present invention is not limited to this, and the image to be corrected need not be an image captured by the imaging unit 10. In this case, the imaging object detection unit 41 detects a target object to be corrected (in other words, the target area including the target object) included in the image. In other words, the imaging object detection unit 41 functions as a target area detection unit detecting the target area including the imaging object included in the image or the target object other than the imaging object (in other words, the target area included in the image).
The correction area configuration unit 42 configures, in the input image, the correction area to be corrected, based on information indicating the imaging object detected by the imaging object detection unit 41 (example: information indicating the position and size of the imaging object). The correction area configuration unit 42 configures an area including the imaging object included in the input image, in the input image as the reduction area (first area). The correction area configuration unit 42 configures, in the input image, an area outside the configured reduction area, specifically, an area adjacent to the reduction area, as an enlargement area (second area). In other words, the correction area configuration unit 42 configures the reduction area and the enlargement area as a correction area. Details will be described below.
The correction intensity configuration unit 43 configures an enlargement ratio and a reduction ratio for image correction. The correction intensity configuration unit 43 may use preset values as a scaling factor α (<1)) for reducing the reduction area configured in the input image and a scaling factor β (>1) for enlarging the enlargement area configured in the input image, or may use values configured in accordance with the input image or a user operation.
The clipping configuration unit 44 configures a clipping area to be clipped from the image to be finally output, based on the position and size of the reduction area configured by the correction area configuration unit 42. Details will be described below.
The image correction unit 45 reduces the reduction area, enlarges the enlargement area, and clips the clipping area to correct the input image and outputs the corrected input image to the printing unit 50 and/or the display unit 30 as an output image.
Note that the image correction unit 45 may use preset values as a first scaling factor (<1) for reducing the reduction area configured in the input image and a second scaling factor (>1) for enlarging the enlargement area configured in the input image, or may use values configured in accordance with the input image or the user operation.
Now, a flow of image processing in the controller 40 of the image printing apparatus 1 according to the present embodiment will be described using
In a case of acquiring the image captured by the imaging unit 10 as an input image, the imaging object detection unit 41 detects an imaging object to be corrected from the input image (S201: a target area detection step). The correction area configuration unit 42 configures the reduction area and the enlargement area, based on information indicated by the imaging object detected by the imaging object detection unit 41 (S202: a correction processing step). The correction intensity configuration unit 43 configures a correction intensity (scaling factor α of reduction in the reduction area and the scaling factor β of enlargement in the enlargement area), based on information indicating the reduction area and the enlargement area configured by the correction area configuration unit 42 (S203: a correction processing step). The clipping configuration unit 44 configures a clipping area, based on information indicating the reduction area and the enlargement area configured by the correction area configuration unit 42 and information indicating the correction intensity configured by the correction intensity configuration unit 43 (S204: a correction processing step). The image correction unit 45 reduces the reduction area, enlarges the enlargement area, and clips the clipping area to generate an output image (S205: a correction processing step). The controller 40 outputs the generated output image to the printing unit 50 or the display unit 30.
Now, the details of the correction processing by the controller 40 will be described with processing examples.
(a) of
(b) of
The correction area configuration unit 42 configures an area outside the reduction area 302 as an enlargement area 303. In Processing Example 1, the enlargement areas 303 are provided at opposing positions across and adjacent to reduction area 302. In Processing Example 1, the width of the each of the enlargement areas 303 is half of the width of the reduction area 302, but the present embodiment is not limited to this.
In Processing Example 1, the image correction unit 45 reduces the reduction area 302 in the lateral direction, and enlarges the enlargement areas 303 in the lateral direction. More specifically, assume that coordinates in a direction indicated by an arrow in
For example, the correspondence relationship between the x-coordinate in the input image 300 and the x-coordinate in the output image is represented by a graph as illustrated in (c) of
On the other hand, the scaling factor β is configured to be greater than 1, and thus, the slope of the x-coordinate of the output image with respect to the x-coordinate of the input image 300 is also greater than 1. This means that in the enlargement area 303, enlargement occurs toward the centerline 304 of the reduction area 302 and that the degree of the enlargement (moving distance of pixels) decreases consistently with the distance from the centerline 304 increases.
The x-coordinate of the input image coincides with the x-coordinate of the output image at the end of the enlargement area 303 opposite to the reduction area 302. In this way, the area outside the enlargement area 303 in the input image 300 is not enlarged or reduced. Thus, the image correction unit 45 can generate an output image while preventing image quality from being degraded.
Note that it is sufficient that the image correction unit 45 specifies, for the reduction area 302, the amount of correction for each point P according to a distance r from the centerline 304 to the point P and that the image correction unit 45 need not make the amount of correction for each point P proportional to the distance r from a correction center c to the point P as described above.
Now, a case will be described where configuration of a sufficient correction area in the input image is precluded in a case that the imaging object to be corrected is located at the image end or that the size of the correction target is large compared to the image. (a) of
(b) of
The correction area configuration unit 42 configures enlargement areas 403 such that the enlargement areas 403 are located opposite to each other across and adjacent to the reduction area 402. In Reference Example 1, the width of the enlargement area 403 on the left side of the reduction area 402 is half of the width of the reduction area 302 as in Processing Example 1. However, a right end of the input image 400 is located close to a right end of the reduction area 402, and thus, the width of the enlargement area 403 on the right side of the reduction area 402 is smaller than the width in Processing Example 1.
In Reference Example 1, in a case that correction processing is performed using the reduction ratio and the enlargement ratio configured under conditions similar to the conditions in Processing Example 1, the relationship between the x-coordinate of the input image 400 and the x-coordinate of the output image is represented by a graph illustrated in (c) of
In contrast, even in a case that the sufficient correction area fails to be configured, the controller 40 according to the present embodiment can generate, by appropriately clipping the end portion side of the output image, a preferable output image with the image quality of the image end prevented from being degraded while correcting the correction area with a pre-configured correction intensity.
(a) of
(b) of
In this way, in a case that a sufficient enlargement area can be configured, the controller 40 can enlarge the enlargement area to compensate for the effect of reduction of the reduction area, thus generating an output image while preventing deterioration of the image quality as in Processing Example 1. That is, the area outside the enlargement area in the image can be an area that is not reduced or enlarged. In a case that a sufficient enlargement area fails to be configured, the controller 40 can configure and clip the clipping area to obtain a preferable output image including no area having indefinite pixel values as in Processing Example 2.
To achieve such an effect, the clipping configuration unit 44 performs the following operation. In a case that the reduction direction of the reduction area extends from one side toward the inside of the reduction area, the clipping configuration unit 44 determines whether to clip a first side of the image depending on a distance from the reduction area to the first side. In a case that the first side is to be clipped, the clipping configuration unit 44 configures a clipping area on the first side. That is, in a case that the space between the reduction area (first area) and one side (first side) of the input image is narrow (e.g., in a case the reduction area contacts one side of the input image), the clipping configuration unit 44 configures a clipping area so as to clip the first side of the image, and otherwise configures no clipping area. The clipping area may be, for example, the one-side end portion of the corrected image.
Note that “clipping a certain side of an image” means removing, from the image, an area having a certain width from the certain side included in the image. The “reduction direction of the reduction area (first area)” refers to the direction in which the pixels in the reduction area move in a case that the reduction area is reduced. The “reduction direction of the reduction area extends from the one side toward the inside of the reduction area” means that the one-side pixels in the reduction area move toward the inside the reduction area. In this case, in a case that the reduction direction is represented by a vector and broken down into a vector in a certain direction and a vector in a direction orthogonal to the certain direction, the resultant vector in the certain direction is in a positive direction.
In a case that the reduction direction of the reduction area extends from a certain side toward the inside of the reduction area, the reduction area is reduced so as to move away from the certain side, and thus a sufficient enlargement area is preferably configured on the certain side of the reduction area in order to compensate for the effect of the reduction. However, in a case that the space between the reduction area and the certain side is narrow (for example, in a case that the reduction area contacts the certain side of the input image), a sufficient enlargement area fails to be configured. In such a case, as described in Reference Example 1, an area having indefinite pixel values is formed on the certain side of the reduction area. Thus, the clipping configuration unit 44 configures the clipping area so as to clip the certain side of the image, thus allowing a preferable output image to be obtained that includes no area having indefinite pixel values.
On the other hand, in a case that the space between the reduction area and the certain side is wide, a sufficient enlargement area can be configured. Thus, the clipping configuration unit 44 can generate an output image while preventing the image quality from being degraded without configuring the clipping area.
In other words, the clipping configuration unit 44 is configured such that, in a case that the reduction direction of the reduction area (the first area) extends from a certain side toward the inside of the reduction area, the clipping configuration unit 44 determines whether to clip the certain side of the corrected image depending on the distance from the reduction area to the certain side. This allows the image to be more appropriately corrected.
In an embodiment, the clipping configuration unit 44 is configured as follows. Assume that, in a case that the reduction direction of the reduction area extends from a certain side toward the inside of the reduction area, a distance from the center of the reduction area (first area) to a portion of the reduction area closest to the certain side is d1, a distance from the reduction area to the certain side (from the portion of the reduction area closest to the certain side to the certain side) is dx, an enlargement/reduction ratio for the reduction area is α, and an enlargement/reduction ratio for the enlargement area is β, the clipping configuration unit 44 configures a clipping area on the certain side of the image in a case that following Equation (1) is satisfied, and configures no clipping area in a case that Equation (1) is not satisfied.
(1−α)d1>(β−1)dx (1)
More particularly, in an embodiment, the clipping configuration unit 44 configures a clipping area with the width (1−α)d1−(β−1)dx on the certain side of the corrected image in a case that Equation (1) is satisfied. Accordingly, the controller 40 can clip the appropriate area from the corrected image. Note that the width of the clipping area configured by the clipping configuration unit 44 may be smaller than (1−α)d1−(β-1)dx and that, even in this case, a preferable output image can be obtained that includes a reduced area having indefinite pixel value. The width of the clipping area configured by the clipping configuration unit 44 may be greater than (1−α)d1−(β−1)dx, and even in this case, a preferable output image can be obtained that includes no area having indefinite pixel values. However, the clipping configuration unit 44 preferably provides the clipping area outside the reduced reduction area.
Note that in the processing example described above, the configuration has been described in which the reduction area is configured as a rectangular area traversing the input image in the vertical direction and is reduced in the lateral direction. However, the present embodiment is not limited to this. A correction method is not particularly limited, and various methods can be used, for example, (A) a configuration in which reduction and enlargement occur toward a specific straight line (configurations in Processing Examples 1 and 2 described above) and (B) a configuration in which reduction and enlargement occur toward a specific point (configurations in Processing Examples 3 to 5 described below). (A) In the configuration in which reduction and enlargement occur toward a specific straight line, the straight lines may be the center line of the reduction area or any other straight line, and the direction of the straight line may be the vertical direction, the lateral direction, or an oblique direction in the image. The shape of the reduction area and the enlargement area is not limited to a rectangle, and may partly include a curve. (B) In the configuration in which reduction and enlargement occur toward a specific point, the point may be the center of the reduction area or any other point, and the aspect of enlargement and reduction may be isotropic or anisotropic. The shape of the reduction area and the enlargement area is not limited to a circle, and may be an ellipse or a polygon. Hereinafter, as an example of (B) the configuration in which reduction and enlargement occur toward a specific point, a configuration in which reduction and enlargement occur toward the center of a circular reduction area will be described with processing examples.
(a) of
The correction that the image correction unit 45 applies to the input image is a correction in which (1) the reduction area 551 is reduced toward the correction center c, and in which (2) the enlargement area 552 is enlarged toward the correction center c. More specifically, this correction involves transferring a point P=(r cos θ, r sin θ)+(c1, c2), where r represents a distance from the correction center c and θ represents a direction viewed from the correction center c=(c1, c2), to (1) a point P′=(r′ cos θ, r′ sin θ)+(c1, c2) in a case of r≤d1, where the distance from the correction center c is r′=αr and the direction viewed from the correction center c is θ, and to (2) a point P′=(r′ cos θ, r′ sin θ)+(c1, c2) in a case of d1<r≤d2, where the distance from the correction center c is r′=βr−(β−a)d1 and the direction viewed from the correction center c is θ. Here, α represents a positive constant configured by the correction intensity configuration unit 43 as an enlargement/reduction ratio for the reduction area 551, and satisfies α<1. On the other hand, β represents a positive constant configured by the correction intensity configuration unit 43 as an enlargement/reduction ratio for the enlargement area 552, is defined by β=(d2−αd1)/(d2−d1), and satisfies β>1.
For example, in a case of α=0.9 and β=1.1 (d2=2d1), a relationship between the distance r from the correction center c to an uncorrected point P and the distance r′ from the correction center c to a corrected point P is represented by a graph illustrated in (b) of
Note that it is sufficient that the image correction unit 45 specifies, for the reduction area 551, the amount of correction for each point P according to the distance r from the correction center c to the point P and that the image correction unit 45 need not make the amount of correction for each point P proportional to the distance r from the correction center c to the point P as described above.
As described above, the image correction unit 45 may further change the amount of correction for each point P according to the angle θ to reduce the reduction area 551 in an anisotropic manner.
(a) of
(b) of
Thus, the clipping configuration unit 44 configures, instead of the area 607, a rectangular area 603 including the area 607 as a clipping area. Thus, as illustrated in (c) of
Now, a case will be described in which the target area to be corrected is located in a corner of the input image. (a) of
(b) of
Thus, the clipping configuration unit 44 configures, as clipping areas, two rectangular areas 703 ((b) and (d) of
As described above, in the present embodiment, in a case that the correction area is located at the image end and the reference destination of a part of the output image is located outside the input image, the output image is clipped so as to exclude areas having indefinite pixel values due to a reference to the outside of the image, thus allowing a preferable corrected image to be obtained that includes no area having indefinite pixel values.
In particular, correction can be performed with a pre-configured correction intensity regardless of the position of the correction area. This allows the effect of the correction to be made constant regardless of the position of the correction area. Thus, in a case that faces of two persons are present at different positions in the image, e.g., at the center of the image and at the right end of the image, and that correction is performed to reduce each face, similar effects can preferably be exerted on the two persons.
Another aspect of the present embodiment may be an image capturing apparatus including no printing function.
Another embodiment of the present invention (Embodiment 2) will be described below, based on
In Embodiment 2, the controller 40 further clips an image resulting from clipping of an area having indefinite pixel values to generate a preferable output image having an aspect ratio unchanged after the correction. The correction processing according to the present embodiment will be described below with processing examples.
(a) of
(b) of
Thus, in the present embodiment, the clipping configuration unit 44 configures, in addition to the rectangular area 805 including the area 802 having indefinite pixel values, a clipping area 806 on an upper side of the image 800, as illustrated in (d) of
The unchanged aspect ratio produces the following effects. In a case that an image is displayed on a display apparatus such as a liquid crystal display, the entire image can be displayed as long as a display screen has the same aspect ratio as that of the image. On the other hand, in a case that the display screen does not have the same aspect ratio as that of the displayed image, no image is displayed in a part of the display screen (blank area). In a case that the images have different aspect ratios due to image correction (clipping), sequential display of multiple images on the display screen disadvantageously leads to a change in blank area each time the image is switched. However, in a case that multiple displayed images have the same aspect ratio, the blank area preferably remains unchanged in a case that the displayed image is switched. Not only in the case that the image is displayed on the display screen but also in a case that a printing apparatus is used to print the image on paper or as a photograph, printing can preferably be achieved without any blank area as long as the aspect ratio of print paper is the same as the aspect ratio of the image.
Note that the controller 40 need not necessarily perform the correction steps in the order of (a) to (c) of
As described above, in a case of having configured the clipping area on a certain side of the input image, the clipping configuration unit 44 clips a side of the input image (second side) that does not oppose the certain side (first side) such that the aspect ratio of the output image resulting from correction is the same as the aspect ratio of the uncorrected input image. This allows the output image that maintains the aspect ratio of the image to be obtained.
In an embodiment, assume that the number of horizontal pixels in the uncorrected input image is nx, the number of vertical pixels in the uncorrected input image is ny, and the width of the clipping area is w, (i) in a case that the target area is located close to the right side or left side of the input image and the clipping area is configured on the right side or left side of the input image, the clipping configuration unit 44 may further configure a clipping area with a width w(ny/nx) on the upper side or the lower side, and (ii) in a case that the target area is located close to the upper side or lower side of the input image and the clipping area is configured on the upper side or lower side of the input image, the clipping configuration unit 44 may further configure a clipping area with a width w(nx/ny) on the right side or the left side.
Now, a method for configuring a position where the image is clipped to adjust the aspect ratio will be described. (a) of
Here, in the present embodiment, in addition to the rectangular area 905 including the area 902 having indefinite pixel values, the clipping configuration unit 44 further configures a clipping area on the upper side or the lower side that does not oppose the right side on which the area 902 is configured in the image 900 such that the aspect ratio of the output image is the same as the aspect ratio of the input image.
In this case, in Processing Example 7, the clipping configuration unit 44 configures an area 906 on the lower side of the image 900 as a clipping area, as illustrated in (c) of
Note that the criteria for selecting the clipping area from the candidate areas for the clipping area for the clipping configuration unit 44 to not change the aspect ratio of the image before and after correction is not limited to the criteria whether the face of the person is included. For example, in a case that no face of a person is contained in candidate areas for the clipping area for preventing the aspect ratio from being changed after correction, the clipping configuration unit 44 may determine whether each of the candidate areas contains an imaging object of interest other than persons and may configure, as a clipping area, a candidate area containing no imaging object of interest. In a case that none of the candidate areas contain a face of a person or an imaging object of interest, the clipping configuration unit 44 may calculate a feature amount for each candidate area and clip to exclude candidate areas with small feature amounts. The clipping configuration unit 44 can calculate the feature amount by using color information, for example, edge detection and saturation.
In other words, in a case of configuring a clipping area with a specific width (first width) on the upper side or the lower side (or the right side or the left side) to prevent the aspect ratio of the image from being changed after correction, the clipping configuration unit 44 configures a clipping area on one of the upper side and the lower side (or the right side and the left side) that contains no pixel of interest within a specific width from this side. Thus, the clipping configuration unit 44 can avoid losing important portions of the output image.
Here, the pixel of interest means a person, a feature point for detecting an imaging object of interest, or a pixel for detecting a feature amount.
Now, as a method for configuring the position at which the image is clipped to adjust the aspect ratio, a method based on the position of a face of a person may be used. (a) of
Here, in the present embodiment, in addition to the rectangular area 1005 including the area 1002 having indefinite pixel values, the clipping configuration unit 44 further configures a clipping area on the upper side or the lower side that does not oppose the right side on which the area 1002 is configured in the image 1000 such that the aspect ratio of the output image is the same as the aspect ratio of the input image.
In this case, in Processing Example 8, the clipping configuration unit 44 configures, as a clipping area, an area 1006 on the upper side of the image 1000 that is a side farther from the reduction area including the person 1001, as illustrated in (d) of
Note that in a case that the clipping configuration unit 44 configures an area 1007 on the lower side of the image 1000, which is a side closer to the reduction area including the person 1001, as the clipping area, as illustrated in (e) of
Another embodiment (Embodiment 3) of the present invention will be described below, based on
In Embodiment 3, the controller 40 performs a pixel number conversion on an image with the aspect ratio adjusted to generate a more preferable output image. The correction processing according to the present embodiment will be described below with processing examples.
(a) of
(b) of
Thus, in the present embodiment, in a case that clipping the clipping area that is the area 1104 having indefinite pixel values and clipping the clipping area 1105 to adjust the aspect ratio lead to the numbers of vertical and horizontal pixels in the image differing from the numbers of vertical and horizontal pixels in the input image, the image correction unit 45 enlarges the image 1102 resulting from clipping of the clipping area such that the input image and the output image are equal in the numbers of vertical and horizontal pixels.
Image enlargement processing can be performed by a known interpolation method such as a nearest-neighbor method, a bilinear method, a bicubic method, or the like. (c) of
Note that the controller 40 need not necessarily perform the adjustment of the aspect ratio based on clipping and the pixel number conversion in the order of (a) to (c) of
Another embodiment (Embodiment 4) of the present invention will be described below, based on
The controller 40 enlarges a partial area of an image resulting from clipping of an area having indefinite pixel values to generate a preferable output image with the aspect ratio of the image unchanged after correction. The correction processing according to the present embodiment will be described below in detail with processing examples.
(a) of
(b) of
Thus, in the present embodiment, the correction area configuration unit 42 configures a second enlargement area (third area) 1506 on a side (left side, third side) opposite, with respect to the reduction area 1511, to a side (right side, first side) from which an image has been clipped such that the aspect ratio of the output image is the same as the aspect ratio of the input image, as illustrated in (b) of
Accordingly, unlike in Embodiment 2, the aspect ratio can be adjusted without unnecessary clipping. Note that the second enlargement area 1506 is preferably wider. A wider second enlargement area 1506 makes the enlargement/reduction ratio β for the second enlargement area 1506 close to 1, allowing degradation of the quality of the output image to be suppressed.
The correction area configuration unit 42 preferably specifies, as the second enlargement area 1506, a part of the area located on the side opposite, with respect to the reduction area, to the side of the reduction area from which the image has been clipped, the part including no pixel of interest. This allows avoidance of containment of a face of a person and the like in the second enlargement area 1506 leading to deformation of the face and the like.
Another embodiment (Embodiment 5) of the present invention will be described below, based on
In a case that the reduction area is located closer to a certain side (first side) of the input image to prevent an enlargement area with a sufficient width from being configured on the first side of the reduction area, the image processing apparatus according to the present embodiment changes the reduction direction of the reduction area to avoid formation of an area having indefinite pixel values. This differs from the configuration of a clipping area on the first side by the image processing apparatus according to Embodiments 1 to 4.
The imaging object detection unit 141 has functions similar to the imaging object detection unit 41. The correction area configuration unit 142 has functions similar to the correction area configuration unit 42. The correction direction configuration unit 143 has functions similar to the correction intensity configuration unit 43, and further includes a function to change the reduction direction of the reduction area and the enlargement area, and a function to configure the clipping area, as described below. The image correction unit 144 has functions equivalent to the image correction unit 45.
Now, a flow of image processing in the controller 140 according to the present embodiment will be described by using
In a case of acquiring an image captured by the imaging unit 10 as an input image, the imaging object detection unit 141 detects an imaging object to be corrected from the input image (S201: a target area detection step). The correction area configuration unit 142 configures the reduction area and the enlargement area, based on information indicated by an imaging object detected by the imaging object detection unit 41 (S202: a correction processing step). The correction direction configuration unit 143 configures, based on the information indicated by the reduction area and the enlargement area configured by the correction area configuration unit 42, a correction intensity (a scaling factor α for the reduction of the reduction area and a scaling factor β for enlargement of the enlargement area) and performs some of change of the reduction direction of the reduction area, change of the enlargement area, and configuration of the clipping area as necessary (S1603: a correction processing step). The image correction unit 144 reduces the reduction area, enlarges the enlargement area, and clips the clipping area as necessary to generate an output image (S205: a correction processing step). The controller 40 outputs the generated output image to the printing unit 50 or the display unit 30.
Now, the details of the correction processing by the controller 140 will be described with processing examples.
(a) of
In this case, the correction direction configuration unit 143 changes the reduction direction of the reduction area 1702 so as to prevent the direction from extending from the right side of the input image 1700 toward the inside of the reduction area 1702. For example, the correction direction configuration unit 143 changes the reduction direction of the reduction area 1702 such that the direction extends toward the side (the right side of the input image 1700) located in proximity to the reduction area 1702. The correction direction configuration unit 143 reconfigures the enlargement area so as to avoid configuring the enlargement area on the side (right side of the input image 1700) located in proximity to the reduction area 1702 with respect to the reduction area 1702.
(b) of
Here, the reduction direction of the reduction area 1702 is rightward, and thus a correction is performed to transfer a point P=(e−1, y), where 1 is a distance from the right end x=e of the reduction area 1702, to a point P′=(e−α1, y), and the right end of the corrected reduction area 1706 is also x=e. This eliminates a need to configure the enlargement area on the right side of the reduction area 1702. Thus, as in Reference Example 1, even in a case that the reduction area is located closer to a certain side (first side) of the input image to prevent a sufficient enlargement area from being configured on the first side of the reduction area, formation, on the first side of the reduction area, of an area having indefinite pixel values can be avoided.
Note that, assume that the left end of the uncorrected reduction area 1702 is x=e−w, the left end of the corrected reduction area 1706 is x=e−αw, which is shifted rightward from x=e−w and that the image correction unit 144 thus shifts the enlargement area 1707 and an area on the left side of the enlargement area 1707 rightward. As a result, a shaded area 1704 located at the left end of an output image 1704 resulting from correction has indefinite pixel values.
Thus, the correction direction configuration unit 143 configures the area 1704 as a clipping area. As a result, as illustrated in (c) of
Note that instead of configuring the area 1704 as a clipping area, the correction direction configuration unit 143 can enlarge an area between the area 1704 and the enlargement area 1707 so as to supplement the area 1704, thus obtaining a preferable output image including no area having indefinite pixel values.
As described above, in a case that the reduction direction of the reduction area extends from one side (first side) of the input image toward the inside of the reduction area, the correction direction configuration unit 143 in the present embodiment determines whether to change the reduction direction of the reduction area depending on the distance from the reduction area to the first side. In a case of changing the reduction direction of the reduction area, the correction direction configuration unit 143 changes the reduction direction of the reduction area so as to prevent the reduction direction from extending from the first side toward the inside of the reduction area. Thus, a preferable output image can be obtained that includes no area having indefinite pixel values.
(a) of
In this case, as is the case with Processing Example 11, the correction direction configuration unit 143 changes the reduction direction of the reduction area 1802 so as to prevent the reduction direction from extending from the right side of the input image 1800 toward the inside of the reduction area 1802, and reconfigures the enlargement area so as to avoid configuring the enlargement area on the side (right side of the input image 1800) located in proximity to the reduction area 1802 with respect to the reduction area 1802. Furthermore, the correction direction configuration unit 143 increases the enlargement ratio for the enlargement area 1803.
(b) of
Control blocks of the controller (image processing apparatus) 40 and 140 (in particular, the imaging object detection unit 41, the correction area configuration unit 42, the correction intensity configuration unit 43, the clipping configuration unit 44, the image correction unit 45, the imaging object detection unit 141, the correction area configuration unit 142, the correction direction configuration unit 143, and the image correction unit 144) may be implemented by a logic circuit (hardware) formed in, for example, an integrated circuit (IC chip) such as an Application Specific Integrated Circuit (ASIC), or a Field Programmable Gate Array (FPGA), or may be implemented by software by using a Central Processing Unit (CPU) or a Graphics Processing Unit (GPU).
In the case of software implementation, the controllers (image processing apparatuses) 40 and 140 include a CPU performing instructions of a program that is software implementing the functions, a Read Only Memory (ROM) or a storage apparatus (these are referred to as “recording media”) in which the program and various data are stored so as to be readable by a computer (or CPU), a Random Access Memory (RAM) in which the program is decompressed, and the like. The computer (or CPU) reads from the recording medium and performs the program to achieve the object of the present invention. As the above-described recording medium, a “non-transitory tangible medium” such as a tape, a disk, a card, a semiconductor memory, and a programmable logic circuit can be used. The above-described program may be supplied to the above-described computer via an arbitrary transmission medium (such as a communication network and a broadcast wave) capable of transmitting the program. Note that one aspect of the present invention may also be implemented in a form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.
An image processing apparatus (controller 40) according to Aspect 1 of the present invention includes a correction processing unit configured to reduce a first area (reduction area) in an image and to enlarge a second area (enlargement area) outside the first area to correct the image (correction area configuration unit 42, correction intensity configuration unit 43, clipping configuration unit 44, and image correction unit 45), and in a case that a reduction direction of the first area extends from a first side of the image toward an inside of the first area, the correction processing unit determines whether to clip the first side of the image depending on a distance from the first area to the first side.
In the configuration described above, in a case that the reduction direction of the first area (reduction area) extends from the first side of the image toward the inside of the first area, and the distance from the first area to the first side is short, preventing a second area (enlargement area) with a sufficient width from being provided on the first side of the first area, an area having indefinite pixel values may be formed on the first side of the output image. Thus, by clipping the first side of the image depending on the distance from the first area to the first side, a preferable output image can be obtained that includes no area having indefinite pixel values.
The image processing apparatus according to Aspect 2 of the present invention includes Aspect 1 described above, wherein the correction processing unit may be configured to clip the first side of the image in a case that the first area and the first side adjoin.
According to the above-described configuration, even in a case that the first area (reduction area) contacts the first side of the input image and the second area (enlargement area) fails to be configured on the first side of the first area (reduction area), a preferable output image can be obtained that includes no area having indefinite pixel values.
The image processing apparatus according to Aspect 3 of the present invention includes Aspect 1 or 2 described above, wherein the correction processing unit may be configured to clip the first side of the image in a case that a distance from the first area to the first side is shorter than a width of the second area opposite to the first side across the first area.
According to the above-described configuration, even in a case that the first area (reduction area) is located close to the first side of the input image to prevent the second area (enlargement area) with a sufficient width from being configured on the first side of the first area (reduction area), a preferable output image can be obtained that includes no area having indefinite pixel values.
The image processing apparatus according to Aspect 4 of the present invention includes Aspects 1 to 3 described above, wherein the correction processing unit may be configured to clip a second side of the image that does not oppose the first side such that an aspect ratio of the image after correction is the same as an aspect ratio of the image before the correction.
According to the above-described configuration, an output image with the aspect ratio of the image maintained can be obtained.
The image processing apparatus according to Aspect 5 of the present invention includes Aspect 4, wherein the correction processing unit may be configured to clip as many as a first width of the second side and to specify, as the second side, one of sides of the image that do not oppose the first side of the image, with no pixel of interest included within the first width from the one side.
According to the above-described configuration, losing important portions of the output image can be avoided.
The image processing apparatus according to Aspect 6 of the present invention includes Aspect 4 described above, wherein the correction processing unit may be configured to specify, as the second side, one of sides of the image that do not oppose the first side, the one side being located farther from the first area.
According to the above-described configuration, in the output image, an object to serve as an object of interest can be located closer to the center of the image.
The image processing apparatus according to Aspect 7 of the present invention includes any one of Aspects 4 to 6 described above, wherein the correction processing unit may be configured to enlarge the image as a whole resulting from clipping of the first side and the second side such that the number of vertical pixels and the number of horizontal pixels in the image after correction are identical to the number of vertical pixels and the number of horizontal pixels in the image before the correction.
According to the above-described configuration, an output image can be obtained that maintains the numbers of vertical and horizontal pixels in the image.
The image processing apparatus according to Aspect 8 of the present invention includes Aspects 1 to 3 described above, wherein the correction processing unit may be configured to enlarge a third area located on a third side opposite to the first side with respect to the first area such that an aspect ratio of the image after correction is identical to an aspect ratio of the image before the correction.
According to the above-described configuration, an output image with the aspect ratio of the image maintained can be obtained.
The image processing apparatus according to Aspect 9 of the present invention includes Aspect 8 described above, wherein the correction processing unit may be configured to specify, as the third area, a part of an area located on the third side with respect to the first area, the part including no pixel of interest.
According to the above-described configuration, a high-quality output image can be obtained that maintains the aspect ratio with important portions prevented from being deformed.
An image processing apparatus (controller 140) according to Aspect 10 of the present invention includes a correction processing unit configured to reduce a first area (reduction area) in an image and to enlarge a second area (enlargement area) outside the first area to correct the image (correction area configuration unit 142, correction direction configuration unit 143, and image correction unit 144). In a case that a reduction direction of the first area extends from a first side of the image toward an inside of the first area, the correction processing unit determines whether to change the reduction direction of the first area to prevent the reduction direction from extending from the first side of the image toward the inside of the first area depending on a distance from the first area to the first side.
In the configuration described above, in a case that the reduction direction of the first area (reduction area) extends from the first side of the image toward the inside of the first area, and the distance from the first area to the first side is short, preventing a second area (enlargement area) with a sufficient width from being provided on the first side of the first area, an area having indefinite pixel values may be formed on the first side of the output image. Thus, depending on the distance from the first area to the first side, a preferable output image including no area having indefinite pixel values can be obtained by changing the reduction direction of the first area to prevent the reduction direction from extending from the first side of the image toward the inside of the first area.
An imaging apparatus 2 according to an Aspect 11 of the present invention includes an imaging unit 10, and the image processing apparatus according to any one of Aspects 1 to 10 described above, the image processing apparatus being configured to correct the image captured by the imaging unit 10.
According to the above-described configuration, a user can capture an image of a face of a person and preferably perform image processing on the captured image.
An image printing apparatus according to Aspect 12 of the present invention includes the image processing apparatus according to any one of Aspects 1 to 10 described above, and a printing unit 50 configured to print the image corrected by the image processing apparatus.
According to the above-described configuration, the user can easily print an image obtained by performing the image processing.
An image printing apparatus 1 according to Aspect 13 of the present invention includes an imaging unit 10, the image processing apparatus according to any one of Aspects 1 to 10 described above, the image processing apparatus being configured to perform image processing on the image captured by the imaging unit 10, and a printing unit 50 configured to print the image corrected by the image processing apparatus.
According to the above-described configuration, the user can easily print an image obtained by performing image processing on the captured image.
A control method for an image processing apparatus according to Aspect 14 of the present invention includes the step of correcting an image by reducing a first area in the image and enlarging a second area outside the first area, and the step of correcting the image includes determining, in a case that a reduction direction of the first area extends from the first side toward an inside of the first area, whether to clip the first side of the image depending on a distance from the first area to the first side.
A control method for an image processing apparatus according to Aspect 15 of the present invention includes the step of correcting an image by reducing a first area in the image and enlarging a second area outside the first area, wherein the step of correcting the image includes determining, in a case that a reduction direction of the first area extends from the first side toward an inside of the first area, whether to change the reduction direction of the first area to prevent the reduction direction from extending from the first side of the image toward the inside of the first area depending on a distance from the first area to the first side.
According to the above-described configurations, effects similar to the effects of the image processing apparatus according to an aspect of the present invention are exerted.
The image processing apparatus according to each of the aspects of the present invention may be implemented by a computer. In this case, the present invention embraces also an image processing program that implements the above-described image processing apparatus by a computer by causing the computer to operate as each of the units (software elements) included in the above-described image processing apparatus, and a computer-readable recording medium in which the program is recorded.
The present invention is not limited to each of the above-described embodiments. It is possible to make various modifications within the scope of the claims. An embodiment obtained by appropriately combining technical elements each disclosed in different embodiments falls also within the technical scope of the present invention. Further, a combination of technical elements disclosed in the respective embodiments allows formation of a new technical feature.
This application claims the benefit of priority to JP 2017-084003 filed on Apr. 20, 2017, which is incorporated herein by reference in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2017-084003 | Apr 2017 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2018/001063 | 1/16/2018 | WO | 00 |
