Patent Grant
11062122
The present invention relates to an image candidate determination apparatus, an image candidate determination method, a program for controlling the image candidate determination apparatus, and a recording medium storing the program.
At events such as graduation ceremonies or sports day, many children are imaged, and albums or the like are created. For this purpose, for example, a school photo management server capable of centrally managing, browsing, and ordering all photos taken in a school, student's works, or the like is considered (JP2014-102715A). Further, in creating an album, a method for preventing the number of photographic images taken by a specific individual from becoming extremely large or small is considered. For this purpose, images are selected from list images, a person is extracted from the selected images, the number of appearances of the extracted person is counted, and in a case where the number of appearances exceeds five, a warning is displayed. In a case where the warning is displayed, a user selects an image to be deleted among the selected images (JP2010-057073A). Further, an image publishing device capable of automatically determining, even though a user does not set a public state or a private state, the public state or the private state according to a predetermined rule is also considered (JP2013-191035A). Furthermore, a method for imaging a plurality of subjects in a balanced manner is also considered (JP5616819B).
Energetic children or conspicuous children appear in many photos, while shy children or quiet children do not appear in the photos. In the technique disclosed in JP2014-102715A, there is a possibility that in a case where photos are sold in such a state, complaints of unfairness may be raised from guardians. In the technique disclosed in JP2010-057073A, in a case where an image to be deleted is selected from selected images, it is difficult to determine which image is to be selected. In the technique disclosed in JP2013-191035A, images are determined to be made public or private on the basis of an image analysis result, in which the number of images including a specific person may increase, or may decrease. Further, in the technique disclosed in JP5616819B, a plurality of subjects are merely imaged in a balanced manner during imaging.
An object of the invention is to provide a technique that assists which image is to be selected in order to uniformize the number of images that includes each person in images that is made public as much as possible.
According to an aspect of the invention, there is provided an image candidate determination apparatus comprising: image grouping means for receiving a plurality of images and grouping one or more images in which the same person is included; and image candidate determination means for determining an extraction image candidate or a non-extraction image candidate from the images grouped by the image grouping means, on the basis of a total image evaluation value and a limit value of an extraction number of images in which the same person is included.
Here, the grouping means may be any means as long as the image candidate determination apparatus can recognize one or more images in which the same person is included. For example, the grouping means may be stored in a certain folder, or a certain flag may be set therefor.
Further, the image candidate determination apparatus according to the invention may comprise a processor that receives a plurality of images, groups one or more images in which the same person is included, and determines an extraction image candidate or a non-extraction image candidate from the grouped images, on the basis of a total image evaluation value and a limit value of an extraction number of images in which the same person is included.
According to another aspect of the invention, there is provided an image candidate determination method suitable for the image candidate determination apparatus. That is, the method comprises receiving a plurality of images and grouping one or more images in which the same person is included, by image grouping means; and determining an extraction image candidate or a non-extraction image candidate from the images grouped by the image grouping means, on the basis of a total image evaluation value and a limit value of an extraction number of images in which the same person is included, by image candidate determination means.
Further, according to still another aspect of the invention, there are provided a computer-readable program for controlling a computer of the image candidate determination apparatus, and a recording medium storing the program.
The image candidate determination apparatus may further comprise an image candidate display control means that displays the extraction image candidate or the non-extraction image candidate determined by the image candidate determination means on the display screen.
The total image evaluation value may be calculated on the basis of a person image evaluation value calculated for each person who appears in the images, for example.
The person image evaluation value is calculated on the basis of an evaluation value relating to at least a face, for example. In this case, the image candidate determination apparatus further comprises face image portion extraction means for extracting a face image portion from each of a plurality of images, the image grouping means groups the face image portions extracted by the face image portion extraction means for each face image portion of the same person, and the image candidate determination means determines the extraction image candidate or the non-extraction image candidate from the images grouped by the image grouping means, for example, on the basis of the total image evaluation value and the limit value of the extraction number of images in which the same person is included.
The image candidate determination apparatus may further comprise correction means for correcting the person image evaluation value using a correction coefficient for setting the person image evaluation value to be larger as the number of images in which the same person is included is smaller.
The image candidate determination apparatus may further comprise image determination means for determining an extraction image or a non-extraction image from the extraction image candidates displayed under control of the image candidate display control means, or determining a non-extraction image or an extraction image from the non-extraction image candidates displayed under control of the image candidate display control means.
The image candidate determination apparatus may further comprise notification means for notifying the number of the extraction images and the number of the non-extraction images determined by the image determination means, for each person included in the images.
The image candidate determination apparatus may further comprise notification means for updating and notifying, in a case where a plurality of images are defined as extraction images or non-extraction images that are set in advance for each person included in the images, the number of extraction images and the number of non-extraction images in accordance with the determination of the extraction images or the non-extraction images by the image determination means, for each person included in the images.
Further, the image candidate determination apparatus may further comprise image candidate stopping means for excluding, in a case where an extraction number of images in which a specific person is included deviates from a limit value as one non-extraction image candidate determined by the image candidate determination means becomes a non-extraction image or one extraction image candidate becomes an extraction image determined by the image candidate determination means, the one non-extraction image candidate or the one extraction image candidate from the non-extraction image candidates or the extraction image candidates.
The image grouping means regards non-same person as the same person for adults, for example, and performs grouping for respective images.
The image candidate determination apparatus may further comprise first evaluation value calculating means for calculating a lower person image evaluation value as the number of persons included in an image is larger.
For example, the image candidate determination means may determine, on the basis of the total image evaluation value and the limit value of the extraction number of images in which the same person is included, the extraction image candidate or the non-extraction image candidate from the images grouped by the image grouping means so that an extraction number of images in which the same person is included is within the limit value.
The image candidate determination apparatus may further comprise person detecting means for detecting a person for which the number of the same person included in one or more images grouped by the image grouping means is equal to or smaller than a threshold value, and imaging instruction means for performing an imaging instruction of the person detected by the person detecting means.
According to the invention, a plurality of images are received, and one or more images in which the same person is included are grouped. On the basis of a total image evaluation value and a limit value (the limit value refers to an upper limit value, a lower limit value, or the like) of an extraction number of images in which the same person is included, an extraction image candidate (the extraction image candidate refers to a candidate for an image suitable for a user to extract from the images) or a non-extraction image candidate (the non-extraction image candidate refers to a candidate for an image suitable for the user not to extract from the images) is determined. Since the extraction image candidate or the non-extraction image candidate is determined, the user can understand which image is preferably extracted or which image is preferably not extracted. For this reason, the user can designate an image to be extracted from the images that are preferably considered to be extracted, or can designate an image that is not to be extracted from the images that are not preferably considered to be extracted.
An overall operation of the image candidate determination apparatus 1 is controlled by a central processing unit (CPU) 2.
The image candidate determination apparatus 1 includes a display device 3 that displays images and other information on a display screen, and a communication device 4 that communicates with devices other than the image candidate determination apparatus 1 by being connected to the Internet and other networks. Further, the image candidate determination apparatus 1 also includes a hard disk 5, a hard disk drive 6 that accesses the hard disk 5, a memory 7 that stores data or the like, and a keyboard 8 and a mouse 9 for inputting commands or the like. In addition, the image candidate determination apparatus 1 includes a compact disc drive 10 that accesses a compact disc 11, and a memory card reader/writer 12 that writes data on a memory card 13 and reads data recorded on the memory card 13.
An operation program of the image candidate determination apparatus 1 to be described later is received by the communication device 4 through the Internet. The received operation program is installed in the image candidate determination apparatus 1. The operation program may be received by the image candidate determination apparatus 1 through a network such as the Internet, may be recorded on a portable recording medium such as the compact disc 11 without being installed in the image candidate determination apparatus 1, and then, may be read from the portable recording medium. In this case, the operation program read from the portable recording medium is installed in the image candidate determination apparatus 1. It goes without saying that the operation program is readable by a computer (CPU 2) of the image candidate determination apparatus 1.
In this embodiment, in a case where a large number of images including a person are captured and the images are made public, such an imbalance that the number of images of a specific person is large or small is eliminated as much as possible.
In this embodiment, names of persons that are imaging targets are registered in the image candidate determination apparatus 1 by a user of the image candidate determination apparatus 1. In the embodiment to be described below, a large number of images are captured at a kindergarten graduation ceremony. In this embodiment, names of children that are imaging targets are registered in the image candidate determination apparatus 1 in advance by a user (for example, a kindergarten teacher).
In a case where the memory card 13 in which image data indicating a plurality of images is stored is loaded in the image candidate determination apparatus 1, the image data indicating the plurality of images stored in the memory card 13 is read by the memory card reader/writer 12 (step S21 in
An image display window 80 shown in
Referring to
An event of “graduation ceremony” is generated in the event management area 82, and the images represented by the image data read from the memory card 13 are managed as the event of “graduation ceremony”. Specifically, in a case where the event addition button 83 is clicked after the character string of “kindergarten” is clicked, an event input window appears on the image display window 80, and the user inputs a new event using the event input window. In a case where the user inputs the character string of “graduation ceremony” using a keyboard 8 at that time, the character string of “graduation ceremony” is displayed below the character string of “kindergarten”.
As the images represented by the image data read from the memory card 13 are dragged and dropped in the image display area 90 in a state where the character string of “graduation ceremony” is clicked, the dragged and dropped images are managed under the event of “graduation ceremony” (this state is the state shown in
A left arrow button 92 and a right arrow button 93 are formed under the image management tab 91. An image specifying number 94 for specifying a designated image is displayed on a right side of the right arrow button 93. A numeral represented by the image specifying number 94 represents the designated image. In the example shown in
On a right side of the image specifying number 94, a character string 95 of “list display” is displayed. In a case where a check box on a left side of the character string 95 of “list display” is checked (the check box is checked in an initial state), a plurality of images are displayed in the image display area 90 as a list, as shown in
Under each of the plurality of images displayed in the image display area 90, a public status and an image file name are displayed. In this embodiment, there are three types of public statuses such as “public”, “private”, and “hold”. The “public” among the public statuses represents a state where an image is released, the “private” among the public statuses represents a state where an image is not released, and the “hold” among the public statuses represents a state where either the “public” or “private” is not determined. For example, a user creates an album using only “public” images, or prints only the “public” images.
At a right end portion of the image display area 90, an up arrow button 97, a slide bar 98, and a down arrow button 99 are formed. As the up arrow button 97 is clicked or the slide bar 98 is dragged upward, the image display area 90 is scrolled upward, and as the down arrow button 99 is clicked or the slide bar 98 is dragged downward, the image display area 90 is scrolled downward.
In a case where a certain image among the images displayed in the image display area 90 is clicked (YES in step S26 of
In a case where the left arrow button 92 or the right arrow button 93 is clicked (YES in step S28 of
In a case where the check box of the character string 95 in the list display is unchecked (YES in step S30 in
In
A character string 96 of “face information display” is displayed on a right side of the character string 95 of “list display”. A check box is also formed on a left side of the character string 96 of “face information display”.
In the example shown in
In a case where the check box on the left side of the “face information display” character string 96 is checked (YES in step S32 of
Faces surrounded by face frames F1, F4, F3, and F2 respectively correspond to persons (students) with names “Taro SATO”, “Jiro TANAKA”, “Hanako ITO”, and “Haruko SUZUKI”, and the names are respectively displayed near (above) the respective face frames F1, F4, F3, and F2. Since a face surrounded by a face frame F is not given association with a name by the user (unclassified), the name is not displayed. As the check box on the left side of the character string 96 of face information display is checked, the name of the person included in the image is displayed, and thus, it is possible for the user to confirm the name of the person included in the image.
In a case where the check box on the left side of the face information display character string 96 is unchecked (YES in step S34 of
In a case where the face classification tab 101 is clicked by the user (YES in step S36 of
In a case where the face classification tab 101 is clicked by the user, an image display window 80 shown in
In the image display area 90, face image portions extracted in the process of step S22 of
A subject list area 110 is formed on a left side of the image display area 90. In the subject list area 110, names that are registered in advance in the image candidate determination apparatus 1 by the user as described above are displayed as a list. Further, the number of images in which a person with a name is shown is also displayed in correspondence with the name. The name displayed in the subject list area 110 may be selected (clicked) by the user.
Above the image display area 90, a face classification button 111 in which a “face classification” character string is displayed, a “selected student display” character string 112, a “non-selected student display” character string 113, and a “unclassified face display” character string 114 are displayed. Check boxes are respectively formed on left sides of the “selected student display” character string 112, the “non-selected student display” character string 113, and the “unclassified face display” character string 114.
In a case where the face classification button 111 is clicked, face image portions of the same person are collectively displayed (to be adjacent to each other) in the image display area 90. In a case where the check box on the left side of the “selected student display” character string 112 is checked, a face image of a person with a selected name among the names displayed in the subject list area 110 is displayed in the image display area 90. For example, in a case where the check box on the left side of the “selected student display” character string 112 is checked in a state where “Taro SATO” among the names displayed in the subject list area 110 is selected, a face image of “Taro SATO” is displayed in the image display area 90.
In a case where the check box on the left side of the “non-selected student display” character string 113 is checked, face images of persons other than the selected name among the names displayed in the subject list area 110 are displayed in the image display area 90. For example, in a case where the check box on the left side of the “non-selected student display” character string 113 is checked in a state where “Taro SATO” among the names displayed in the subject list area 110 is selected, face images of persons other than “Taro SATO” are displayed in the image display area 90. In a case where the check box on the left side of the “unclassified face display” character string 114 is checked, face images of unclassified persons (whose names and face images are not associated by the user, as described above) are displayed in the image display area 90.
In a case where the face classification tab 101 is clicked and the face image portions displayed in the image display area 90 are displayed, the user may associate faces with names. As information on a name is stored in association with position information on a face image portion in a header of an image file indicating an image including the face image portion, the face and the name are associated with each other. With respect to a face image portion that is not associated with a name among the face image portions displayed in the image display area 90, only a file name of an image including the face image portion is displayed. With respect to the face image portion associated with the name, a file name of an image including the face image portion and the name are displayed.
In a case where names are associated with the face image portions displayed in the image display area 90, any one name among the names displayed in the subject list area 110 is clicked by the user (YES in step S39 of
Subsequently, a first image evaluation value with which a name is associated is calculated by the CPU 2 (step S42 of
The first image evaluation value is calculated according to evaluation parameters such as the number of persons included in an image and the size of a face included in the image. Table 1 is an example of a person number score table based on the number of persons included in an image.
As the number of persons included in an image is not too small and not too large, it may be considered that the image is well-balanced and the persons included in the image are appropriately conspicuous and evaluation of the image for the persons is high. For this reason, the calculation is performed by the CPU 2 so that the person number score becomes large in a case where the number of persons included in an image is about 5 and the person number score becomes small in a case where the number of persons included in the image is small or large (first image evaluation value calculating means). However, as the number of persons included in the image becomes smaller, the person number score may be set to become higher, and as the number of persons included in the image becomes larger, the person number score may be set to become higher.
Table 2 is an example of a face size score table based on the size of a face included in an image. In Table 2, the size of the face represents the size of the face with respect to the image. For example, in a case where the size of the face is “1/15”, it means that the size of the face is 1/15 of the size of the image. In a case where a face is surrounded by a rectangular frame, the size may be determined using a horizontal pixel count of an image and a horizontal pixel count of a face image portion, using a vertical pixel count of the image and a vertical pixel count of the face image portion, or using the number of pixels in an entire image area and the number of pixels in a face area.
Since the balance of an image is not good either in a case where the size of the face of a person included in the image is too large or in a case where the size of the face of the person included in the image is too small, the face size score is the highest at an appropriate ratio (here, 1/6), and the face size score is set to become lower as a distance therefrom becomes large.
With respect to the evaluation parameters, an expression score calculated by the number of persons included in an image and the size of a face included in the image, and additionally, the degree of focus of a face image portion included in the image, the degree of eye opening in the face included in the image, a facial expression of the face included in the image, and the like may be used, so that the face score may be calculated. A facial expression score is set corresponding to the evaluation parameters so that the facial expression score is higher as the image portion of the face is in better focus, the facial expression score is higher as the eyes are more properly opened, and the facial expression score is higher as the facial expression is more smiling.
In this embodiment, a raw score of the first image evaluation value is calculated on the basis of the sum of the person number score, the face size score, and the facial expression score.
In addition, an image quality score may be defined so that the score becomes higher as the image quality of one entire image becomes better (for example, as the focus becomes better), and may be used to calculate the raw score.
Further, in this embodiment, the first image evaluation value is corrected according to the number of images in which a specific person is shown. For this purpose, weighting is performed so that the first image evaluation value for the person becomes higher as the number of images that include a specific person becomes larger and the first image evaluation value for the person becomes lower as the number of the images that include the specific person becomes smaller.
Table 3 is an example of a weighting table. In Table 3, a maximum value represents a maximum value of images in which the same person is shown among images to be made public, and is “6”, for example.
It is assumed that the first image evaluation value for each person is calculated for each image.
Table 4 shows a first image evaluation value for each image of “Taro SATO”.
Table 4 shows a first image evaluation image obtained from “Taro SATO” for six images of image file names “0022.jpg”, “0019.jpg”, “0021.jpg”, “0013.jpg”, “0020.jpg”, and “0018.jpg”. Here, it is assumed that “Taro SATO” is shown in the six images. With respect to “Taro SATO”, for example, a raw score of the first image evaluation value of the file name “0022.jpg” is “40”, and raw scores of the other first image evaluation values are also shown. Scores obtained by normalizing these raw scores and weighting the normalized raw scores are respective first image evaluation values for “Taro SATO”. Since “Taro Sato” appears in six images (the same number of images as the maximum value defined in the weighting table), a weighting coefficient is “×1” of the maximum value, and a first image evaluation value after correction based on the weighting coefficient becomes equal to the normalized raw score. In a case where the weighting coefficient is “×1”, the first image evaluation value after correction based on the weighting coefficient and the normalized raw score become different from each other. As described above, it is preferable to correct the first image evaluation value by the CPU 2 (correction means) using a weighting coefficient (correction coefficient) for setting the first image evaluation value to is larger as the number of images in which the same person is included is smaller and setting the first image evaluation value to become smaller as the number of images in which the same person is included is larger.
With respect to a person with another name, in addition to “Taro SATO”, similarly, an image evaluation value for the person is calculated for each image.
Table 5 shows, for six images in which “Taro SATO” is included, first image evaluation values for persons included in the six images, and shows a second image evaluation value that is a total image evaluation value for one image (the second image evaluation value corresponds to a total image evaluation value of the invention).
It can be understood that the image of the image file name “0022.jpg” includes “Hanako SUZUKI” in addition to “Taro SATO”, the image of the image file name “0019.jpg” includes “Haruko ITO” in addition to “Taro SATO”, the image of the image file name “0021.jpg” includes “Haruko SUZUKI” in addition to “Taro SATO”, the image of the image file name “0013.jpg” includes “Hanako ITO” in addition to “Taro SATO”, the image of the image file name “0020.jpg” includes “Haruko SUZUKI”, “Hanako ITO”, “Jiro TANAKA”, and “Ichiro WATANABE” in addition to “Taro SATO”, and the image of the image file name “0018.jpg” includes “Ichiro WATANABE” in addition to “Taro SATO”.
For example, the first image evaluation value of the image of the image file name “0022.jpg” is “20” for “Taro SATO”, but is “200” for “Haruko SUZUKI”, and the second image evaluation value that is the total image evaluation value of the image of the image file name “0022.jpg” is “220” that is the sum of “20” that is the first image evaluation value for “Taro SATO” and “200” that is the first image evaluation value for “Haruko SUZUKI”. Similarly, with respect to the other images, it can be understood that the second image evaluation value of the image of the image file name “0019.jpg” is “62.5”, the second image evaluation value of the image of the image file name “0021.jpg” is “367”, the second image evaluation value of the image of the image file name “0013.jpg” is “63”, the second image evaluation value of the image with the image file name “0020.jpg” is “1023”, and the second image evaluation value of the image of the image file name “0018.jpg” is “414.5”. Accordingly, among the six images, the image of the image file name “0020.jpg” having the highest second image evaluation value has the highest importance as the image, and the image of the image file name “0019.jpg” having the lowest second image evaluation value has the lowest importance as the image. In this way, it is preferable to add up the first image evaluation values that are different for each person included in the image for each image, and calculating the sum obtained by the addition by the CPU 2 (second image evaluation value calculating means) to set the sum as the second image evaluation value.
The second image evaluation value may be calculated by any calculation method in which the second image evaluation value is larger as the first image evaluation value is larger on the basis of the first image evaluation value that is calculated for each person who appears in an image, and may be calculated by the sum of the first image evaluation values after weighting. For example, with respect to image groups in events in which entire children participate, taking into account the number of opportunities for future imaging, large weights may be given to the first image evaluation values for children at higher grades (accordingly, small weights may be given for children at lower grades), and thus, the second image evaluation value may be calculated by the sum of the first image evaluation values.
In this embodiment, in a case where private images are determined from the six images, images having the low second image evaluation values are preferentially determined as private image candidates (non-extraction image candidates), and in a case where public images are determined therefrom, images with the high second image evaluation values are preferentially determined as public image candidates (extraction image candidates). In a case where an image is changed to be made public or private, the first image evaluation value and the second image evaluation value are calculated again.
In the above-described embodiment, in a case where an image includes a plurality of face image portions, the sum of evaluation values of the face image portions may be calculated, and the calculated sum may be used as a total image evaluation value. For example, referring to
In a case where an arrow button of the left arrow button 92 or the right arrow button 93 is clicked (YES in step S43 of
In a case where the face classification button 111 is clicked (YES in step S45 of
Referring to
Further, a face image group G2 includes a face image portion included in an image with an image file name “0027.jpg”, a face image portion included in the image with the image file name “0014.jpg”, and a face image portion included in an image with an image file name “0028.jpg”. The face image portions included in the face image group G2 also represent the same person. Similarly, a face image group G3 includes a face image portion included in the image with the image file name “0027.jpg” and a face image portion included in the image with the image file name “0014.jpg”. The face image portions included in the face image group G3 also represent the same person.
Similar to the image display window 80 in the state shown in
Further, in a case where the image management tab 91 is clicked (YES in step S47 of
In
An image specified by the image specifying number 94 before the public setting tab 102 is clicked is displayed in the image display area 90. In
The public/private information display area 120 is formed on a left side of the image display area 90. Names of subjects are displayed in the public/private information display area 120. Check boxes 121 to 126, and 129 are formed on left sides of the names of the subjects, for example. “Teacher/guardian” is displayed on a right side of the check box 129 as a person name. This is because, in the case of a kindergarten graduation ceremony, since students are leading characters and teachers or guardians are supporting characters, the teachers or guardians are treated as the same person without distinction of persons. As described above, adults such as teachers and guardians may be regarded as the same person, and may be grouped for each image by the CPU 2 (image grouping means). The teachers and guardians may be separately treated for each person instead of being regarded as the same person. Check boxes of names corresponding to the persons (persons with the names) included in the image displayed in the image display area 90 are checked. In the example shown in
Further, on the right sides of the names of the subjects, the number of sheets in a state of being set to “private” (the number of sheets of non-extraction images), the number of sheets in a state of being set to “hold”, and the number of sheets in a state of being set to “public” (the number of sheets of extraction images), among images included in the subject, are displayed in the subject list area 110 of the image display window 80 displayed in the display device 3 (notification means) under the control of the CPU 2 (display control means) for each person. The display device 3 (notification means) can notify the number of the at least one extraction image and the number of the at least one non-extraction image, for each person included in the images.
In a case where the image displayed in the image display area 90 is clicked (YES in step S50 of
In the case of two kinds of “private” and “public” without including “hold” in the public status, whenever the image included in the image display area 90 is clicked once, the public status is switched between “private” and “public”, the number of images of “private” and the number of images of “public” that are displayed in the public/private information display area 120 are updated by the CPU 2 for each person included in the image, and are displayed (notified) in the public/private information display area 120 of the image display window 80 displayed in the display device 3 (notification means).
In a case where any arrow button of the left arrow button 92 or the right arrow button 93 is clicked (YES in step S52 of
The image displayed in the image display area 90 includes “Taro SATO” and “Haruko SUZUKI”. The face of “Taro SATO” is surrounded by the face frame F1, and the name “Taro Sato” is displayed above the face frame F1. Similarly, the face of “Haruko SUZUKI” is surrounded by the face frame F2, and the name “Haruko SUZUKI” is displayed above the face frame F2.
In a case where the right arrow button 93 is clicked in a state where the image display window 80 shown in
Referring to
In a case where the right arrow button 93 is clicked in a state where the image display window 80 shown in
Referring to
In a case where the right arrow button 93 is clicked in a state where the image display window 80 shown in
Referring to
In a case where the right arrow button 93 is clicked in a state where the image display window 80 shown in
Referring to
In a case where the right arrow button 93 is clicked in a state where the image display window 80 shown in
Referring to
In a case where the left arrow button 92 is clicked in a state where the image display window 80 shown in
As shown in
For example, as shown in
In a case where a sheet number input button 130 is clicked (YES in step S54 of
Referring to
In a case where the minimum number is input in the minimum sheet number input area 141 and the maximum number is input in the maximum sheet number input area 142 (either the minimum number or the maximum number may be input) (step S56 of
The CPU 2 confirms whether or not there are images that include the same person, in which the number of images that are made public is greater than the input maximum number (step S61 of
For example, it is assumed that images including “Taro SATO” are six images including file names “0022.jpg”, “0019.jpg”, “0021.jpg”, “0013.jpg”, “0020.jpg”, and “0018.jpg”. Then, the six second image evaluation values become values shown in Table 5, for example. Further, it is assumed that the input maximum number is 5. Then, one of the six images is determined as a private image candidate by the CPU 2. As can be understood from Table 5, since an image with the lowest second image evaluation value among the second image evaluation values of the six images is an image having the image file name “0019.jpg”, the image having the image file name “0019.jpg” is determined as a private image by the CPU 2 (image candidate determination means). In this way, a private image candidate (non-extraction image candidate) is determined from grouped images based on the second image evaluation value and the limit value of the extraction number of images including the same person. The CPU 2 (image candidate determination means) can determine an extraction image candidate and a non-extraction image candidate from the grouped images, on the basis of a total image evaluation value and a limit value of an extraction number of images in which the same person is included.
Conversely, five images among the six images in a descending order of the second image evaluation values may be determined as public image candidates (extraction image candidates) by the CPU 2 (image candidate determination means). For example, a window containing a character string of “It is recommended that the image with the image file name “0019.jpg” is made private” may be displayed on the display screen together with the image (non-extraction image candidate) having the image file name “0019.jpg” under the control of the CPU 2 (image candidate display control means). Conversely, the images (extraction image candidates) having the image file names other than the image file name “0019.jpg”, among six images having the file names “0022.jpg”, “0019.jpg”, “0021.jpg”, “0013.jpg”, “0020.jpg”, and “0018.jpg”, may be displayed on the display screen together with a character string of “It is recommended that the images are made public” under the control of the CPU 2 (image candidate display control means). As the user clicks the left arrow button 92 or the right arrow button 93, the image display window 80 in which the image having the image file name “0019.jpg” is displayed in the image display area 90 as shown in
In a case where the user confirms the image displayed in the image display area 90 and makes the image private, the user clicks the image to determine the public status to be private by the CPU 2 (image determination means). Thus, the private image is designated (YES in step S64 and step S65 of
In the above-described embodiment, the private image candidate is determined, but instead of determining the private image candidate, the public image candidate may be determined. For example, the images with the image file names “0022.jpg”, “0021.jpg”, “0013.jpg”, “0020.jpg”, and “0018.jpg” may be used as public image candidates. In a case where an image to be made public from public image candidates is determined, by clicking the image, the public status is determined to be public by the CPU 2 (image determination means).
Further, in determining a private image candidate or a public image candidate, in a case where an extraction number of images including a specific person exceeds the maximum number or is smaller than the minimum number as the determined private image candidate is set to be a private image or the determined public image candidate is set to be a public image (in a case where the extraction number of the images including the specific person is out of the limit value as the determined one non-extraction image candidate is set to be a non-extraction image or the determined one extraction image candidate is set to be an extraction image), it is preferable that such an image is excluded from the private image candidates (non-extraction image candidates) or the public image candidates (extraction image candidates) under the control of the CPU 2 (image candidate stopping means). For example, in Table 5, in a case where the image with the image file name “0019.jpg” is determined as a private image as described above, the number of public images of a specific person (for example, “Hanako ITO”) is smaller than the minimum number, it is preferable that the image with the image file name “0019.jpg” is not determined as a private image candidate, and instead, another image is determined as a private image candidate, or it is preferable that the image with the image file name “0019.jpg” is determined as a public image candidate.
In a case where the end command of the program is not given to the image candidate determination apparatus 1 (NO in step S66 of
In a case where there is no image that is made public, in which the number of images including the same person is larger than the maximum number (NO in step S61 of
In the above-described embodiment, a state where an image group capable of being used as public images after an event is ended is settled is assumed. However, the above-described processes may be performed in real time during the event to image a person with a small number of images. In this case, in the process of step S61 of
In the process of step S67 shown in
That is, such a person that the number of the same person included in one or more grouped images is equal to or smaller than a threshold value is detected by the CPU 2 (person detecting means), and an imaging instruction for the person is performed by the CPU 2 (imaging instruction means).
Referring to
In the imaging instruction window 150, a character string of “Images of OO are insufficient” (a name of a person whose number of images is insufficient is written in OO) to inform the user of the name of the person whose number of images is insufficient is displayed, and also, a character string of “Please take one to five more photos” to inform the user of the number of insufficient images (in which the number of insufficient images varies depending on situations, but it is preferable that the number of images after imaging is equal to or larger than the input minimum number and is equal to or smaller than the input maximum number) is displayed.
On a lower right side of the imaging instruction window 150, a confirm button 151 in which “confirm” is displayed is formed. In a case where the user confirms how many images are insufficient, the user clicks the confirm button 151. Then, the imaging instruction window 150 disappears from the display screen of the display device 3.
In a case where there are no images in which the number of images including the same person is smaller than the minimum number of imaging (NO in step S67 of
In a case where a program end command is given to the image candidate determination apparatus 1 (YES in step S69 of
In a case where the public/private setting image is displayed (step S49 in
In the above-described embodiment, in a case where the program end determination process in step S66 of
Further, in the above-described embodiment, a configuration in which image data representing a plurality of images stored in the memory card 13 is read by the memory card reader/writer 12 is shown, but a configuration in which image data representing a plurality of images is stored in a database server different from the image candidate determination apparatus 1 and the images represented by the image data transmitted from the database server through communication of the database server and the image candidate determination apparatus 1 are displayed on the display screen of the display device 3 of the image candidate determination apparatus 1 as described above may be used.
Further, in the above-described embodiment, a configuration in which image data recorded on the memory card 13 is read is shown, but a configuration in which in a case where the communication device 4 of the image candidate determination apparatus 1 is able to communicate with a digital camera (not shown) by wireless communication means, image data representing images captured using the digital camera are transmitted from the digital camera to the image candidate determination apparatus 1 to perform the above-described processes may be used. Similarly, a configuration in which image data representing images captured by a smartphone is transmitted from the smartphone to the communication device 4 of the image candidate determination apparatus 1 to perform the above-described processes may be used. A configuration in which image data representing images captured by a digital camera or a smartphone is transmitted to the image candidate determination apparatus 1 in real time to perform the above-described processes during imaging may be used. Further, a configuration in which the image data representing the images captured by the digital camera or the smartphone is transmitted to the image candidate determination apparatus 1 after all imaging ends may be used.
Further, in the above-described embodiment, a configuration in which in calculating a raw score of the first image evaluation value, an evaluation value related to the face such as a face size score or an expression score or an evaluation value related to a factor other than the face such as a person number score or an image quality score are used is shown, but a configuration in which only the evaluation value related to the face is used as the raw score of the first image evaluation value, or a configuration in which only the evaluation value related to the factor other than the face is used as the raw score of the first image evaluation value may be used.
Further, in the above-described embodiment, a configuration in which the user associates a face image portion with a name while the user views the face image portion is shown, but a configuration in which the user associates the face image with the name of the person of the face in advance, performs comparison with the face image associated with the name to automatically determine whose the face is, and associates the face image portion with the name may be used. In addition, in the above-described embodiment, a configuration in which name association is performed using a face image portion is shown, but a configuration in which in a case where a person has a name tag, the user confirms the name tag instead of the face image portion to associate the face image portion with the name, or a configuration in which the name written on the name tag is automatically analyzed to associate the face image portion with the name may be used.
The user selects images to be collectively associated with a name using the mouse 9. Specifically, the user drags a cursor 171 on face images to be collectively associated with a name using the mouse 9. Then, a frame of the selected images becomes a thick frame 172. In
Subsequently, the name of the selected face images is selected by the user from names displayed in the subject list area 110. Here, it is assumed that the name of the selected face images is “Jiro TANAKA” and “Jiro TANAKA” is selected. Then, a color around the selected name “Jiro TANAKA” is changed.
In
Further, in
Further, the images displayed in the image display area 90 in
In an event management area 82, a sub event of “graduation certificate grant” and a sub event of “director's words” are generated below the event “graduation ceremony”. Images managed as the event “graduation ceremony” are managed as sub events such as “graduation certificate grant” and “director's words”. Further, a sub event addition button 85 in which a character string of “add sub event” is included is formed on a right side of an event addition button 83.
For example, in a case where the sub event addition button 85 is clicked after the character string of “graduation ceremony” is clicked, a sub event input window appears on the image display window 80, and the user inputs a new sub event using the sub event input window. At that time, in a case where the user inputs the character string of “graduation certificate grant” using the keyboard 8, the character string of “graduation certificate grant” is displayed below the character string of “graduation ceremony”. Similarly, a character string of “director's words” is displayed below the character string of “graduation ceremony”.
As the images managed as the event of “graduation ceremony” are dragged and dropped in the image display area 90 in a state where the character string of “graduation certificate grant” is clicked, the dragged and dropped images are managed by the event of “graduation certificate grant”. In a case where an event deletion button 84 is clicked after a character string indicating a sub event displayed in the event management area 82 is clicked, the sub event specified by the clicked character string and images managed by the sub event are deleted.
The public setting tab 102 is clicked on the image display window 80 shown in
A display switching pull-down 160 is formed on an upper right side of the image display area 90. The display switching pull-down 160 is used in displaying the images in the image display area 90 to be divided according to imaging times or the like. As the pull-down button 161 is clicked, a plurality of time-date display switching menus such as a section of “1 hour” or a section of “2 hours” appear, and a display division is selected therefrom by the user. Images are displayed in the image display area 90 in accordance with the display division designated by the display switching pull-down 160.
It is assumed that the section of 1 hour is selected by the display switching pull-down 160. Then, the image display area 90 is divided every hour. In
A display time 163 displayed in the first image display area 90A is displayed in an upper left portion of the first image display area 90A. In
A check box 162 is formed on a left side of the display time 163, and a check box 164 is formed on an upper right side of an image 165 displayed in the first image display area 90A. Both the check boxes 162 and 164 are provided to designate publication of the image. In a case where the check box 162 formed on the left side of the display time 163 is checked, the check box 164 formed on the upper right side of the image 165 (all images 165 displayed in the first image display area 90A) specified by the display time 163 is checked, and publication of the image 165 is designated. In a case where the check box 164 formed on the upper right side of the image 165 is checked, publication of the checked image is designated. In a case where the check box 164 formed on the upper right side of the image 165 is unchecked, the image is set to be private. In a case where the check box 164 is checked, the vicinity of the image 165 is surrounded by a thick frame 166. This is performed to easily distinguish an image for which the check box 164 is checked and an image for which the check box 164 is not checked. Instead, a color of the frame may be changed.
A character string of “12:00” is displayed in the display time 163 of the second image display area 90B, and images 165 captured at a time slot of 12:00 are displayed in the second image display area 90B. The check box 162 is formed on a left side of the display time 163, and the check box 164 is on an upper right side of the image 165. By checking the check box 162 or 164, the checked image is made public in the same manner as described for the images displayed in the first image display area 90A.
The check boxes 195 are also formed on the upper right sides of the images displayed in the image display area 90 of the image display window 80 shown in
A minimum publication number setting area 180 is formed on a left side of the image display area 90. The minimum publication number setting area 180 is provided to set a minimum number of images to be made public. An up arrow button 181 and a down arrow button 182 are formed on a right side of the minimum publication number setting area 180. Whenever the up arrow button 181 is clicked, a numeral displayed in the minimum publication number setting area 180 is increased one by one, and whenever the down arrow button 182 is clicked, the numeral displayed in the minimum publication number setting area 180 is decreased one by one.
A publication number display area 190 is formed on a left side of the image display area 90. The publication number display area 190 is provided to display the number of images to be made public in correspondence with names of persons included in the images to be made public.
In the publication number display area 190, a name of a subject included in an image (for example, Taro SATO), a publication number (for example, 6), and a representative face image 194 of the subject are displayed. Further, in the publication number display area 190, a mark 191 and a line 193 that serve as a guideline for the minimum publication number set in the minimum publication number setting area 180 are also displayed. In addition, a publication number for each name of a subject is indicated by a bar graph (indicated by hatching) 192. As the publication number is smaller, the length of the bar graph 192 is shorter, and as the publication number is larger, the length of the bar graph 192 is longer. As the bar graph 192 extends further rightward, it shows that the publication number is greater. In a case where a right end of the bar graph 192 does not exceed the line 193, the publication number of subject indicated by the bar graph 192 does not reach the minimum publication number, in a case where the right end of the bar graph 192 reaches the line 193, the publication number of the subject indicated by the bar graph 192 coincides with the minimum publication number, and in a case where the right end of the bar graph 192 exceeds the line 193, the publication number of the subject indicated by the bar graph 192 exceeds the minimum publication number. By checking whether or not the right end of the bar graph 192 exceeds the line 193, it is possible to check whether or not the setting is made so that images of the minimum publication number or greater are disclosed, for each subject.
In addition, a mark 197 for calling attention of the user is displayed on the name of the subject corresponding to the image that does not reach the minimum publication number. By viewing the mark 197, images to be made public are selected so as to reach the minimum publication number. A configuration in which the color of the name of the subject corresponding to the image that does not reach the minimum publication number and the color of the name of the subject corresponding to the image that reaches the minimum publication number are changed may be used. Further, the prohibit mark 170 is also displayed in correspondence with the name displayed in the publication number display area 190. It is possible to prevent in advance an image including a private subject from being erroneously set as a public target.
It is assumed that a private subject 198 is included in an image set as a public target. In such a case, the private subject 198 is trimmed from the images. The private subject 198 is removed from the public images.
An image 200 is displayed in the image display area 90. The image 200 includes three persons of subjects 201, 202, and 203. In a case where the image 200 includes the subjects 201, 202, and 203, face images of the subjects 201, 202, and 203 are respectively recognized, and the face images corresponding to the subjects 201, 202, and 203 should be detected. However, there is a case where face images corresponding to the number of subjects are not detected. For example, the face image of the subject 201 is detected and the face frame F1 is displayed and the face image of the subject 203 is detected and the face frame F3 is displayed, but the face image of the subject 202 may not be displayed.
A subject detection number 210 is displayed above the image 200. Two persons are displayed as the number of subjects detected from the image 200. A confirm message 211 indicating whether the number of persons detected from the image 200 is correct is displayed below the image. By viewing the confirm message, the user can recognize that the face that is not detected is included in the image 200. For example, by selecting a name corresponding to a name of the subject 202 of the face that is not detected, for example, from the names displayed in the public/private information display area 120 and clicking the corresponding face, the selected name and the image 200 are associated with each other.
A variety of processing units that execute the above-described processes may include the CPU 2 that executes software to function as the variety of processing units, or may include a programmable logic device of which a circuit configuration is changeable after manufacturing, such as a field-programmable gate array (FPGA), a dedicated electric circuit that is a processor having a circuit configuration that is dedicatedly designed for executing a specific process, such as an application specific integrated circuit (ASIC), or the like.
One processing unit may be configured by one processor among the variety of processors described above, or may be configured by a combination of the same type or different types of two or more processors (for example, a combination of plural FPGAs or a combination of a CPU and an FPGA). As an example in which the plurality of processing units are configured by one processor, first, as represented by a computer such as a client computer or a server, a configuration in which a combination of one or more CPUs and software forms one processor and this processor functions as a plurality of processing units may be employed. Second, as represented by a system on chip (SoC) or the like, a configuration in which a processor for realizing entire functions of a system including a plurality of processing units using one integrated circuit (IC) chip is used may be employed. As described above, the various processing units are configured using one or more of various processors as a hardware structure.
Further, the hardware structure of the variety of processors is, more specifically, electric circuitry in which circuit elements such as semiconductor elements are combined.
The image candidate determination apparatus may be configured by a personal computer, may be configured by a so-called smart device such as a smartphone or a tablet device, or may be configured by a mobile phone such as a feature phone.
| Number | Date | Country | Kind |
|---|---|---|---|
| JP2017-192890 | Oct 2017 | JP | national |
| JP2018-073377 | Apr 2018 | JP | national |
This application is a Continuation of PCT International Application No. PCT/JP2018/032593 filed on Sep. 3, 2018, which claims priority under 35 U.S.C. § 119(a) to Japanese Patent Application No. 2017-192890 filed on Oct. 2, 2017, and Japanese Patent Application No. 2018-073377 filed on Apr. 5, 2018. Each of the above application(s) is hereby expressly incorporated by reference, in its entirety, into the present application.
| Number | Name | Date | Kind |
|---|---|---|---|
| 20110221922 | Matsumoto et al. | Sep 2011 | A1 |
| 20120013764 | Sakai | Jan 2012 | A1 |
| 20120013782 | Mori | Jan 2012 | A1 |
| 20130243273 | Yamaji et al. | Sep 2013 | A1 |
| 20150294137 | Miyazaki | Oct 2015 | A1 |
| 20180164964 | Hori | Jun 2018 | A1 |
| 20200234076 | Katsumata | Jul 2020 | A1 |
| 20200257887 | Noguchi | Aug 2020 | A1 |
| 20200311464 | Raichelgauz | Oct 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 2010-057073 | Mar 2010 | JP |
| 2012-23501 | Feb 2012 | JP |
| 2013-191035 | Sep 2013 | JP |
| 2014-102715 | Jun 2014 | JP |
| 2014-109864 | Jun 2014 | JP |
| 5616819 | Oct 2014 | JP |
| Entry |
|---|
| International Search Report issued in PCT/JP2018/032593; dated Oct. 30, 2018. |
| International Preliminary Report on Patentability and Written Opinion issued in PCT/JP2018/032593; dated Apr. 8, 2020. |
| The extended European search report issued by the European Patent Office dated Oct. 29, 2020, which corresponds to European Patent Application No. 18865004.8-1207 and is related to U.S. Appl. No. 16/819,426. |
| An Office Action; “Notice of Reasons for Refusal”, mailed by the Japanese Patent Office dated May 18, 2021, which corresponds to Japanese Patent Application No. 2019-546580 and is related to U.S. Appl. No. 16/819,426; with English language translation. |
| Number | Date | Country | |
|---|---|---|---|
| 20200257887 A1 | Aug 2020 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2018/032593 | Sep 2018 | US |
| Child | 16819426 | US |
