Patent Application
20210266414
The present invention relates to a technology of identifying an image which can perform printing or display according to an actual size of a subject in an image editing device and an image generation device.
In a method of calculating distance information from an acquired captured image, for example, distance information can be obtained by acquiring captured images at different viewpoints and obtaining a parallax amount from a correlation value or similarity between the captured images. Japanese Patent Laid-Open No. 2011-232330 discloses an imaging device capable of imaging an object using a stereo imaging device and measuring a length between two designated points. In addition, Japanese Patent Laid-Open No. 2019-9511 discloses an imaging device capable of simultaneously acquiring a captured image and subject distance information by obtaining an image according to light fluxes that have passed through different pupil regions in an imaging optical system included in the imaging device.
It is possible to perform display or printing according to an actual size of a subject by simultaneously acquiring a captured image of the subject and distance information thereof. For example, processing of imaging a child that is a subject at different ages and arranging images of the subject at different ages at the time of imaging on a printing mount according to a predetermined actual size ratio may be performed. Accordingly, a user can produce printed matters by which a growth process of the child can be actually felt.
However, a captured image does not necessarily have image information for allowing printing or display according to an actual size of a subject. In a case where a user views a captured image or a thumbnail image, he/she has difficulty identifying whether the image is an image that can be printed or displayed according to the actual size thereof. In addition, if a plurality of images are arranged on one page in an electronic album, it is generally difficult for a user to identify whether an image can be printed or displayed according to the actual size thereof only by viewing the image. If the size of an image is changed from a predetermined size as instructed by a user although the image can be printed according to the actual size thereof, printing according to the actual size cannot be performed.
The present invention provides an image identification device for identifying whether an image can be generated according to an actual size of a subject.
A device of an embodiment of the present invention is an image identification device for identifying a type of image information, including at least one processor and memory holding a program which makes the processor function as: an acquisition unit configured to acquire the image information and information related to the image information; and an identification unit configured to identify the type of the image information using the information related to the image information. The identification unit identifies the image information as information of an image for which an actual size ratio of a subject can be specified if first information for ascertaining a size in a case where a pixel size of the image information is projected to an object side is included in the information related to the image information.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
An image identification device, an image editing device, and an image generation device according to embodiments of the present invention will be described in detail with reference to the drawings. In each embodiment, the image identification device is a device for identifying a type of image information and is applied to the image editing device and the image generation device. The image editing device is a device used by a user for an operation of editing a captured image, and the image generation device is a device for generating an image for printing or an image for display.
In the present embodiment, an example of performing processing of prohibiting calculation of an image size according to an actual size ratio and change to an arbitrary image size based on an operation instruction of a user if a selected image is an image for which the actual size ratio can be calculated is represented.
The printing unit 140 acquires the image data for printing generated by the electronic album editing device 120 and performs processing of printing on predetermined printing paper. The display unit 150 performs processing of displaying an image corresponding to the image data for printing generated by the electronic album editing device 120 and display processing for reporting to a user using the electronic album printing device 100.
The electronic album editing device 120 includes an image identification device 130, an actual size ratio calculation unit 123, an image arrangement unit 124, and a setting value storage unit 125. Various types of setting information used in the electronic album editing device 120 are stored in advance in the setting value storage unit 125.
The image identification device 130 includes an image acquisition unit 131 and the image identification unit 132 for identifying a type of image information. The image identification device 130 identifies whether an image is an image for which an actual size ratio can be calculated (specified) with respect to image information read from the storage unit 110 (hereinafter referred to as a calculable image). For example, the image acquisition unit 131 acquires image information and metadata corresponding to the image information from the storage unit 110. The image identification unit 132 executes processing of identifying whether a target image is an actual size ratio calculable image using the metadata acquired by the image acquisition unit 131. Here, the image information and the metadata corresponding to the image information may be associated and individually stored in the storage unit 110 or may be stored as one file, for example, an image file in the Exif format.
If the target image is identified by the image identification unit 132 as an actual size ratio calculable image, the actual size ratio calculation unit 123 calculates an image size corresponding to the actual size ratio using the metadata acquired by the image acquisition unit 131. With respect to the actual size ratio, an actual size ratio setting value stored in the setting value storage unit 125 can be acquired and used. Alternatively, an input value indicated by a user through a user instruction unit (an operation unit, a touch panel, or the like, which is not shown) may be used as the actual size ratio.
The image storage unit 124 acquires information about a printing paper size from the printing unit 140 and generates image data for printing by performing processing of arranging the image acquired by the image acquisition unit 131 on the printing paper. If the target image is an actual size ratio calculable image identified by the image identification unit 132, the image arrangement unit 124 prohibits change to a size other than the image size using the image size calculated by the actual size ratio calculation unit 123 in principle. In addition, if the image identification unit 132 identifies that the target image is not an actual size ratio calculable image, the image arrangement unit 124 sets an image size using an image size setting value acquired from the setting value storage unit 125 or an input value of a user through the user instruction unit. A position and a segment region at which the image information will be arranged are determined using information acquired by the image arrangement unit 124 from the setting value storage unit 125 or determined using information on a position and a segment region input by the user through the user instruction unit.
The image arrangement unit 124 of the present embodiment prohibits change from an image size calculated by the actual size ratio calculation unit 123 to other image sizes for an actual size ratio calculable image identified by the image identification unit 132. It is possible to print the actual size ratio calculable image according to the actual size ratio by prohibiting changing of an image size for the actual size ratio calculable image. In processing of printing the actual size ratio calculable image, a measure for preventing acquisition of a printing result different from the actual size ratio which is not intended by the user is considered. That is, in a case where the user instructs the electronic album editing device 120 to change the image size for the actual size ratio calculable image, processing of reporting a printing result of a ratio different from the actual size ratio to the user through the display unit 150 is executed. In a case where the user performs instruction for permitting change to an image size at a ratio different from the actual size ratio as a result of the report to the user, the image arrangement unit 124 changes the image size even if the target image is an actual size ratio calculable image.
Next, processing performed by the electronic album editing device 120 will be described with reference to
In S210, the image acquisition unit 131 acquires image information and metadata corresponding to the image information from the storage unit 110. For example, the image information and the metadata may be stored as separate data files and held in the storage unit 110. Alternatively, both the image information and the metadata are stored in one data file and held in the storage unit 110 in consideration of an error in association of the metadata with the image information and efforts required to maintain the metadata. For example, the metadata can be stored in a header region of a data file including the image information.
In S220, the image identification unit 132 identifies the type of the image information using the metadata acquired by the image acquisition unit 131 in S210. In S220, if the metadata includes information by which a size when a pixel size of the image information is projected to an object side can be calculated, the image identification unit 132 identifies the acquired image as an actual size ratio calculable image. That is, if the metadata includes the information by which a size (hereinafter, referred to as a projection size) when the pixel size of the image side is projected to the object side can be calculated, an actual size ratio can be calculated. The information by which the projection size can be calculated will be described later using
In S230, the actual size ratio calculation unit 123 calculates an image size on printing paper according to a predetermined actual size ratio using the metadata acquired by the image acquisition unit 131 in S210 if the target image is an actual size ratio calculable image. Actual size ratio calculation processing will be described in detail later using
A method of calculating a pixel size (projection size) on an object side will be described with reference to
The imaging magnification is provided as a ratio of a distance 341 between a principal point of the imaging optical system 330 on the object side and a subject to a distance 342 between a principal point of the imaging optical system 330 on the image side and the imaging element 320. Meanwhile, the distance 341 may be approximated to a distance from the top of a lens of the imaging optical system 330 closest to the object side. In addition, the distance 342 may be approximated to a focal distance of the imaging optical system 330. That is, the pixel size 343 on the object side can be calculated if the pixel size and the imaging magnification of the imaging element 320 are ascertained. The imaging magnification is calculated on the premise that the distance to the subject and the distance between the principal point of the imaging optical system and the imaging element (or the focal distance of the imaging optical system 330) are ascertained. Distance information of the subject can be calculated from a plurality of pieces of image data with different viewpoints, for example, according to an imaging plane phase difference detection method using a pupil division type imaging element.
Although the method of projecting the pixel size of the imaging element 320 to the object side has been described with reference to
The distance between the imaging optical system 330 and the subject (subject distance) may be a distance to any one point with respect to subject region information included in the image information. Considering a case in which the subject and the imaging device 310 do not face each other, it is desirable that the distance information of the subject be acquired at a plurality of view angles and stored in metadata. Meanwhile, the distance information is depth information representing a depth of the subject in the depth direction in a captured image. The depth information may include an image distortion amount map calculated from a plurality of viewpoint images having different viewpoints, and a defocus amount map calculated by multiplying an image distortion amount by a predetermined conversion factor, for example. A distance map obtained by converting a defocus amount into distance information of the subject and distance image information represent a distance distribution with respect to the captured image.
In addition, as the information (first information) for determining an image for which an actual size ratio can be specified, the number of pixels per unit length may be calculated and recorded as resolution information in the Exif format. Meanwhile, although 1 mm may be used as the unit length, 1 inch may be used and recorded as pixels per inch (PPI) information if printing is considered.
The distance between the imaging optical system 330 and the subject O can be calculated using the following Equation (1) based on an imaging formula of a lens.
b=1/{1/f−1/a} Equation (1)
In Equation (1), b represents a distance between a principal point of the imaging optical system 330 on the object side and the subject O. f represents a focal distance of the imaging optical system 330 and a represents a distance between a principal point of the imaging optical system 330 on the image side and an image point. The distance between the principal point of the imaging optical system 330 on the image side and the image point can be calculated using the distance 342 and a defocus amount (distance between the imaging element 320 and the image point). Accordingly, the defocus amount may be stored in a file as metadata instead of the distance between the imaging optical system 330 and the subject. The defocus amount can be detected by a focus detection unit (not shown) included in the imaging device 310 through a known method.
Next, the actual size ratio calculation processing illustrated in S230 of
S232 is processing of determining whether a target image is an actual size ratio calculable image. In the determination processing, the image identification result in S220 of
In S233, the actual size ratio calculation unit 123 calculates a pixel size on printing paper with respect to the image information acquired in S210 of
In S234, the actual size ratio calculation unit 123 calculates an image size on the printing paper. The actual size ratio calculation unit 123 multiplies the number of horizontal pixels and the number of vertical pixels of the image information by the pixel size on the printing paper calculated in S233. Further, the actual size ratio calculation unit 123 calculates an image size in the horizontal direction and an image size in the vertical direction on the printing paper using the printing information acquired in S231.
After the processing of S234, the processing proceeds to processing of S240 of
If the target image is an actual size ratio calculable image, the image arrangement unit 124 prohibits change to an arbitrary image size in principle. Subsequently, the image arrangement unit 124 sets a segment region of the image information. With respect to the segment region, region information about the segment region can be acquired from the setting value storage unit 125 and used. Alternatively, a value representing a segment region input by the user through the user instruction unit can be used. When the image arrangement unit 124 sets a position with respect to arrangement of the image information, the image arrangement unit 124 acquires a setting value of the position with respect to arrangement from the setting value storage unit 125 and uses the setting value or uses a setting value input by the user through the user instruction unit. Subsequently, the display unit 150 displays an image corresponding to the image information for printing on a screen. The image arrangement unit 124 outputs image data for display corresponding to the image information for printing to the display unit 150, and the display unit 150 acquires the image data for display and performs display processing. When an operation of instructing the image information for printing is performed through the user instruction unit, the image information for printing is decided.
In the present embodiment, it is possible to identify whether an image can be printed or displayed according to an actual size. It is possible to prevent a user from mistakenly editing an image in an image size at a ratio different from an actual size ratio when image information of the image can be printed or displayed according to the actual size.
An example of the electronic album printing device 100 has been described in order to describe functions of the image identification device according to the present invention. The present invention is not limited thereto, and there is an embodiment in which electronic album data generated by the electronic album editing device 120 is transmitted to a server and the server performs processing of printing the electronic album. This applies to the following modified example and embodiment.
Next, a modified example of the first embodiment will be described in detail. In the modified example, in a case where a plurality of images have been selected, processing of selecting only actual size ratio calculable images therefrom and arranging the selected images in a time series is performed.
The image selection unit 426 performs processing of selecting actual size ratio calculable images identified by the image identification device 130. Images for which the user has instructed a selecting operation through the user instruction unit are selected from actual size ratio calculable images. Hereinafter, an image selected by the image selection unit 426 is referred to as a selected image.
The actual size ratio calculation unit 123 calculates an image size according to an actual size ratio using metadata corresponding to the selected images. The image arrangement unit 424 arranges the selected images using the image size calculated by the actual size ratio calculation unit 123 and generates image data for printing. Processing of acquiring information about a date and time or a time of imaging of each selected image from the metadata corresponding to the selected images and arranging image information in a time series is executed.
A method of arranging images in a time series will be described with reference to
In the electronic album editing device 420, the image identification device 130 and the image selection unit 426 select actual size ratio calculable images from a plurality of images. It is difficult for the user to identify whether a target image is an actual size ratio calculable image. Accordingly, in the modified example, the image identification device 130 and the image selection unit 426 automatically select actual size ratio calculable images from a plurality of images. Accordingly, a burden of image selection on the user can be reduced.
According to the modified example, the image arrangement unit 424 can arrange images of the same person on the printing paper 430 illustrated in
The image selection unit 426 can select a desired image of the user from actual size ratio calculable images. For example, if a growth process of the same person is configured as a plurality of images and represented as illustrated in
In addition, the electronic album printing device 400 can automatically determine whether a person corresponding to image information is the same as a person selected by the user by determining whether the image information satisfies a predetermined rule. The predetermined rule may be, for example, a rule for determination using known machine learning. Processing of determining whether a condition that a viewpoint of a captured image is a predetermined viewpoint (a viewpoint at which an optical axis during imaging is approximately parallel to the ground such as in imaging from the front) is satisfied is executed. It is possible to produce a printed matter by which the user can more realistically feel a growth process of the same person by lining viewpoints of captured images up. The printed matter is produced using the printing unit 140 or a printing device included in a server which acquires image data for printing.
Next, a second embodiment of the present invention will be described with reference to
The image selection unit 521 selects only an image identified as an actual size ratio calculable image by the image identification device 130 from a plurality of images acquired by the image acquisition unit 131. That is, an image selected by the image selection unit 521 is a selected image.
The actual size ratio calculation unit 123 calculates an image size according to an actual size ratio on printing paper through processing of S231, S233 and S234 illustrated in
It is difficult to identify an image that can be printed according to an actual size of a subject only by viewing a preview display image or a thumbnail image on the screen of the display unit 150. Accordingly, the printing image editing device 520 automatically selects only an image that can be printed according to the actual size and executes printing processing. As a result, a burden of image selection on the user can be reduced. With respect to the image data for printing generated by the printing image editing device 520, a printed matter is produced according to a printing instruction for the printing unit 140 or a printing instruction for a printing device from a server that acquires the image data for printing.
In the present embodiment, the actual size ratio calculation unit 123 may calculate an image size that becomes a predetermined actual size ratio on the display screen of the display unit 150. In this case, in S231 of
Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the fictions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2020-030699, filed Feb. 26, 2020, which is hereby incorporated by reference wherein in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2020-030699 | Feb 2020 | JP | national |
