Patent Application
20240249579
The present invention relates to an information processing system, an information processing method, and a non-transitory computer-readable storage medium.
A known entry and exit management system for managing entry and exit authorizes users using captured images. In a company and the like, an entry and exit management system installed with multi-stage authorization gates may be used where an authorization gate for entering a building is installed after an authorization gate for entering the premises.
International Publication No. 2018/110012 discloses a technique for reducing processing load is described that enables only people who have entered premises who have had their identification information obtained at an entry gate to be the target of a face image search using facial authentication when entering a restricted area. Also, Japanese Patent Laid-Open No. 2006-309490 discloses a technique for stopping queues being made for an authorization process that permits passage at a predetermined gate regardless of the result of a biometric authentication but performs a more rigorous determination on people who failed the previous authorization than on people who passed the previous authorization when a further authorization is required after passing through the gate.
However, with the technique disclosed in International Publication No. 2018/110012, when authorization at an entry gate where the first stage of authorization is performed fails, the next people are stopped and congestion occurs. For a company and the like, a congestion at a first authorization gate to enter the premises before the start of work can adversely affect the start of work for many people.
Also, with the technique disclosed in Japanese Patent Laid-Open No, 2006-309490, because entry is not restricted when authorization fails at the first stage of authorization, there is a possibility that a normally non-authorized person may be permitted to enter. Thus, there is a security risk.
According to one embodiment of the present disclosure, an information processing system comprises: at least one memory storing instructions; and at least one processor that, upon execution of the instructions, is configured to operate as: first comparing unit configured to compare a feature of a person detected in a first image and a feature of a person registered as a person permitted to pass through a first place, first controlling unit configured to perform control to permit passage through the first place in a case where a first similarity between the feature of the person detected in the first image and the feature of the person registered as the person permitted to pass through the first place is equal to or greater than a first threshold, second comparing unit configured to compare a feature of a person detected in a second image and a feature of a person registered as a person permitted to pass through a second place reached after passing through the first place, and second controlling unit configured to perform control to not permit the person to pass through the second place in a case where a second similarity between the feature of the person detected in the second image and the feature of the person registered as the person permitted to pass through the second place is less than a second threshold greater than the first threshold.
According to another embodiment of the present disclosure, an information processing method comprises: comparing a feature of a person detected in a first image and a feature of a person registered as a person permitted to pass through a first place; performing control to permit passage through the first place in a case where a first similarity between the feature of the person detected in the first image and the feature of the person registered as the person permitted to pass through the first place is equal to or greater than a first threshold; comparing a feature of a person detected in a second image and a feature of a person registered as a person permitted to pass through a second place reached after passing through the first place; and performing control to not permit the person to pass through the second place in a case where a second similarity between the feature of the person detected in the second image and the feature of the person registered as the person permitted to pass through the second place is less than a second threshold greater than the first threshold.
According to yet another embodiment of the present disclosure, a non-transitory computer-readable storage medium stores a program for causing a computer to: compare a feature of a person detected in a first image and a feature of a person registered as a person permitted to pass through a first place; perform control to permit passage through the first place in a case where a first similarity between the feature of the person detected in the first image and the feature of the person registered as the person permitted to pass through the first place is equal to or greater than a first threshold; compare a feature of a person detected in a second image and a feature of a person registered as a person permitted to pass through a second place reached after passing through the first place; and perform control to not permit the person to pass through the second place in a case where a second similarity between the feature of the person detected in the second image and the feature of the person registered as the person permitted to pass through the second place is less than a second threshold greater than the first threshold.
Further features of the present invention will become apparent from the following description of exemplary embodiments (with reference to the attached drawings).
Hereinafter, embodiments will be described in detail with reference to the attached drawings. Note, the following embodiments are not intended to limit the scope of the claimed invention. Multiple features are described in the embodiments, but limitation is not made to an invention that requires all such features, and multiple such features may be combined as appropriate. Furthermore, in the attached drawings, the same reference numerals are given to the same or similar configurations, and redundant description thereof is omitted.
An information processing system according to the first embodiment performs a comparison (first comparison) of a person at a first place and a comparison (second comparison) of a person at a second place reached after passing through the first place. The information processing system, for the first comparison, compares the feature of a person detected from a first image and the feature of a person registered as a person permitted to pass through the first place. In a case where the similarity is equal to or greater than a first threshold, the information processing system performs control to permit passage at the first place. Next, the information processing system, for the second comparison, compares the feature of a person detected from a second image and the feature of a person registered as a person permitted to pass through the second place. In a case where the similarity is less than a second threshold, the information processing system performs control to not permit passage at the second place. Here, the second threshold is greater than the first threshold.
The authorization processing according to the present embodiment is processing to check whether a person (person detected from a captured image) imaged by the image capture apparatus 102 is a person that is registered as a person permitted to pass through each place (gate). The information processing apparatus 105 calculates the similarity of a feature by comparing the feature of a person detected from an image and the feature of a registered person, and determines whether or not to permit the detected person to pass through each place on the basis of the calculated similarity. Note that in the description hereinafter, the place where determination of whether the person is a person permitted to pass through is performed by the information processing apparatus 105 is a gate that can permit people to pass through by opening and closing. However, the place is not limited to this, and it is sufficient that the place can permit a person to pass through. Note that hereinafter, a person that is the target of the authorization processing by the information processing apparatus 105 when passing through a gate in this manner may be referred to as a “person entering” or a “person exiting”.
The information processing apparatus 105 according to the present embodiment calculates the similarity (a first similarity) between the feature of a person imaged by the image capture apparatus 102A at the first gate 101A and the feature of a register person and determines whether or not the calculated similarity is equal to or greater than the first threshold. Next, in a case where the first similarity is equal to or greater than the first threshold, the information processing apparatus 105 performs control to permit passage at the first place.
Note that here, the information processing apparatus 105 according to the present embodiment may additionally determine whether or not the first similarity is equal to or greater than the second threshold (this will be described below in detail). In this case, the information processing apparatus 105 classifies the authorization result as permitted, deferred, or not permitted on the basis of whether the first similarity is equal to or greater than the second threshold, less than the second threshold but equal to or greater than the first threshold, or less than the first threshold. In this example, in a case where the authorization result is permitted or deferred, the information processing apparatus 105 opens the first gate 101A and permits passage. Also, in a case where the compared person is not registered as a person permitted to pass through the first place, the information processing apparatus 105 sets the authorization result as “not permitted”. According to such a configuration, by restricting the passage of people who are “not permitted” and permitting passage of people who are “deferred”, congestion can be reduced while maintaining a certain level of accuracy in the security.
Note that in a case where the authorization result is deferred or not permitted, the information processing apparatus 105 may notify the user of the authorization result using the notification apparatus 103A via audio, lighting, vibration, monitor display, or the like. In the example in
Also, the information processing apparatus 105 according to the present embodiment calculates the similarity (a second similarity) between the feature of a person imaged by the image capture apparatus 102B at the second gate 101B and the feature of a register person and determines whether or not the calculated similarity is less than the second threshold. Next, in a case where the second similarity is less than the second threshold, the information processing apparatus 105 performs control to not permit passage at the second place.
Here, the information processing apparatus 105 can classify the authorization result as permitted or not permitted on the basis of whether the second similarity is equal to or greater than the second threshold or less than the second threshold. In this example, in a case where the authorization result is permitted, the information processing apparatus 105 opens the second gate 101B and permits passage. According to processing in this manner, a more relaxed standard is used to permit passage at the first gate, and a standard which is stricter than the standard used at the first gate is used to permit passage at the second gate, allowing normal security measures to be maintained. The processing for imaging, authorization, gate control, and notification at each gate will be described below in detail.
Note that in the description below, the processing according to the present embodiment is controlled by the information processing apparatus 105. However, the configuration is not particularly limited thereto, and it is sufficient that similar processing can be implemented, for example, by an external apparatus different from the information processing apparatus 105 executing a portion or all of the processing (for example, the image capture apparatus 102 executing the authorization processing described below). Also, the information processing apparatus 105 may be a server, a terminal such as a personal computer, or may indicate a portion or all of an information processing system. In other words, the information processing system according to the present embodiment may indicate the information processing apparatus 105.
The central processing unit (CPU) 201 controls the entire processing executed by the information processing apparatus 105. The read only memory (ROM) 202 stores programs and parameters not required to be changed. The random access memory (RAM) 203 temporarily stores data supplied from an external apparatus or the like. The external storage apparatus 204 is a storage apparatus such as a hard disk or a memory card installed in a fixed manner in the information processing apparatus 105. Note that the external storage apparatus 204 may include a flexible disk (FD), an optical disk such as a compact disk (CD), a magnetic or optical card, an IC card, a memory card, or the like that can be insert and removed from the information processing apparatus 105.
Note that the functions and the processing of the information processing apparatus 105 described below are implemented by the CPU 201 reading out a program stored in the ROM 202 or the like and executing the program.
The input I/F 205 is an interface with an input device 209, such as a pointing device or a keyboard for inputting data, that receives an operation from the user. The output I/F 206 is an interface with the monitor 106 for displaying data held by or supplied by the information processing apparatus 105 or the like. The communication I/F 207 connects to the network 104. The CPU 201 transmits and receives data between the gates 101, the image capture apparatuses 102, and the notification apparatuses 103 via the network 104. The gates 101, the image capture apparatuses 102, and the notification apparatuses 103 have a communication function and connect to the network 104 to communicate with an external apparatus.
The personal ID according to the present embodiment is identification information (ID) that uniquely identifies a person. As the personal ID, an ID used outside of the system, such as a company employee number, may be used unchanged or a separate new ID may be prepared. The authorization state according to the present embodiment indicates the authorization result at each gate for the corresponding person. The authorization state is “unauthorized” when the person is not present at the gate, such as before entering the gate or after exiting the gate and “authorized with deferral” or “authorized” depending on the authorization result at the time of entry when the person has passed through the gate and entered.
The entry state according to the present embodiment is information indicating up to which gate the person has entered. By referencing the entry state, between which gates is the corresponding person located at can be known and which gate the person will likely go to next can be known. As the entry state, for example, “has not entered” is assigned when the person has not passed through the first gate and “first gate” is assigned when the person has passed through the first gate but not passed through the second gate. The information relating to authority to pass through each gate is information indicating whether the person has authority to pass through (whether the person is a person permitted to pass through) each gate and is registered in advance. In the present embodiment, information relating to a person without authority to pass through the first gate is registered in the database and managed. For example, in a case where more than one office uses the same system and the authority to pass through the first gate is set per office, passage authority is also registered per office. For example, a person with an ID may have the authority to pass through the first gate of xx office but not have the authority to pass through the first gate of yy office. Also, for example, there may be cases where, for a visitor or the like that is given temporary passage authority (made a person permitted to pass through), the passage authority is changed after the person exits or the setting is held for a certain time period. Note that the facial feature value of the person used in comparison is obtained by extracting the facial feature from a face imaged in advance and registered.
Note that in the present embodiment, all of the data is stored in one table. However, the data may be managed in a different manner. For example, the database may be divided into one database per gate or divided into a facial feature database to facilitate quick searching using the facial feature as a key. Also, the information processing apparatus 105 may be configured to generate a database for managing the data of only people with “authorized” or “authorized with deferral” as the authorization state upon entry.
First, in step S501, the detection unit 301 detects a region of a person in an image captured by the image capture apparatus 102A. Here, the number of people detected by the detection unit 301 is one, but in a case where a plurality of people are detected, similar processing can be executed for each one. In step S502, the tracking unit 302 tracks the person detected by the detection unit 301 across a plurality of frames. The tracking unit 302 at all times stores a video of a predetermined number (can be set to any number) of the most recent frames while tracking the person. Also, in a case where the person being tracked goes outside the field of view and becomes undetectable, the tracking unit 302 stops tracking the person. The detection processing of a person by the detection unit 301 and the tracking processing of a person by the tracking unit 302 can be executed by typically used known person detection/tracking techniques. Thus, these will not be described in detail here.
In step S503, the target determination unit 303 sets a person in an image as the target person for authorization. In this example, the target determination unit 303 sets a person being tracked that has reached a predetermined position in an image as the target person for authorization. Here, as the predetermined position, a position that is suitable as the position of a target person for authorization at the first gate, for example, in a rectangular region of a predetermined shape with the center of an image as the center position, is set when the information processing system is installed.
In step S504, the authorization unit 304 extracts a feature of the target person. Note that in the present embodiment described below, a facial feature value is extracted from the face region of a person and comparison is performed on the basis of this facial feature value. However, no such limitation is intended, and it is sufficient that a comparison of a person can be performed. Also, the authorization unit 304 extracts the facial feature of the person from each frame of a predetermined number of the most recent stored images. In step S505, the authorization unit 304 compares each facial feature extracted in step S504 with the facial feature registered in the database in advance and outputs the face with the highest similarity from among all of the pairs of registered faces and predetermined number of facial features.
In step S506, the authorization unit 304 determines whether or not the highest similarity (hereinafter, also referred to as the first similarity) corresponding to the face output in step S505 is less than the first threshold (a not permitted threshold TH_NG). In a case where the first similarity is equal to or greater than the not permitted threshold, the processing proceeds to step S507. In a case where the first similarity is less than the not permitted threshold, the processing proceeds to step S515.
In step S507, the authorization unit 304 checks whether the person calculated with the first similarity has the authority to pass through the first gate 101A (in this example, by referencing the database). In step S508, in a case where the person has passage authority, the authorization unit 304 proceeds the processing to step S509. In a case where the person does not have passage authority, the authorization unit 304 proceeds the processing to step S515.
In step S509, the authorization unit 304 determines whether or not the first similarity is equal to or greater than the second threshold (permitted threshold TH_OK). In a case where the first similarity is equal to or greater than the permitted threshold, the processing proceeds to step S510. In a case where the first similarity is less than the permitted threshold, the processing proceeds to step S512.
In a case where the processing arrives at step S510, the authorization result is set to “permitted”, and the notification control unit 305 notifies the person entering that the authorization result is “permitted” via audio or lighting. In step S511, the state management unit 306 changes the authorization state in the database managing the person to “authorized” and the entry state to “first gate” as part of the entry permission processing. Then, the processing proceeds to step S514. In step S514, the gate control unit 307 opens the first gate 101A and ends the processing in
In a case where the processing arrives at step S512, the authorization result is set to “deferred”, and the notification control unit 305 notifies the person entering that the authorization result is “deferred” via audio or lighting. This notification is to make the person entering aware that the authorization result is “deferred” and not “permitted”. Thus, the notification control unit 305 can perform notification control to make the notification corresponding to “permitted” and the notification corresponding to “deferred” different from one another. For example, the notification control unit 305 may not issue a notification in the case of “permitted” and may issue a notification only in the case of “deferred”. Also, the notification control unit 305 may reference a terminal such as a smart device associated with the person entering in the database and notify the person entering by playing a message or audio using the terminal. In step S513, the state management unit 306 changes the authorization state in the database managing the person to “authorized with deferral” and the entry state to “first gate” in entry deferred processing.
In step S514, the gate control unit 307 opens the first gate 101A and ends the processing in
In a case where the processing arrives at step S515, the authorization result is set to “not permitted”, and the notification control unit 305 notifies the person entering that the authorization result is “not permitted” via audio or lighting. In step S516, the state management unit 306 executes entry not permitted processing and ends the processing in
Note that in the example described here, a comparison of a person is performed via a 1: N authentication that calculates the similarity using a person detected in an image and a facial feature registered in the database in advance. However, the method is not particularly limited thereto, and it is sufficient that a comparison of a person can be performed. For example, a comparison of a person may be performed via a 1:1 authentication that compares a person registered in a medium with a person detected in an image, with a reading apparatus (not illustrated) that reads a medium, such as an employee ID card, held by the person being compared being provided on the gate. In this case, whether or not the person entering is registered as a person permitted to pass through can be checked by the reading apparatus reading the medium. In a case where the information processing apparatus 105 determines that the person entering is a person permitted to pass through using the reading apparatus, for example, the information processing apparatus 105 may image the person entering and add a feature of the facial region to the database as a feature of the person permitted to pass through. Also, in a case where a person guided to the reception where an attendant is stationed due to an authorization result of “not permitted” leaves without going to the reception (without being subjected to the checking processing at the reception), the state management unit 306 may execute processing to register an image corresponding to the person to the database on a cautionary list.
Note that the permitted threshold TH_OK (second threshold) and the not permitted threshold TH_NG (first threshold) described above used in the present embodiment are not particularly limited to their values, and it is sufficient that the second threshold is set higher than the first threshold. These thresholds may be set by the user as a desired value depending on the conditions or may be set on the basis of the false rejection rate (FRR) or false acceptance rate (FAR).
Also, in a case where only people with the authority to pass through the first gate are registered in the database, steps S507 and S508 are not performed, and the processing moves to step S509 instead of step S507. In this manner, in the example in
Steps S601 to S604 are similar to steps S501 to S504 in
In step S607, the authorization unit 304 checks whether the person calculated with the second similarity has the authority to pass through the second gate 101B (in this example, by referencing the database). In step S608, in a case where the person has passage authority, the authorization unit 304 proceeds the processing to step S609. In a case where the person does not have passage authority, the authorization unit 304 proceeds the processing to step S612.
In a case where the processing arrives at step S609, the comparison result is set to “permitted”, and the notification control unit 305 notifies the person entering that the comparison result is “permitted” via audio or lighting. In step S610, the state management unit 306 changes the authorization state in the database managing the person to “authorized and deferred” and the entry state to “second gate” as part of the entry permission processing. In step S611, the gate control unit 307 opens the second gate 101B and ends the processing in
In a case where the processing arrives at step S612, the comparison result is set to “not permitted”, and the notification control unit 305 notifies the person entering that the comparison result is “not permitted” via audio or lighting. In step S613, the state management unit 306 executes entry not permitted processing and ends the processing in
In the present embodiment, a plurality of facial features obtained by tracking and a facial feature registered in the database in advance are compared, and the highest similarity from among a plurality of similarities is used. However, the comparison processing is not particularly limited thereto. For example, a predetermined number of people may be registered in the database, and M number of facial features may be obtained by tracking. Then, M number of similarities may be calculated for each person registered in the database. In this case, a comparison using the median value or average value of the calculated similarities may be performed, and any method may be used as long as a person in an image is compared.
Also, in the present embodiment, the same permitted threshold used for comparison at the second gate 101B is used for people given “deferred” and “permitted” at the first gate 101A. However, the permitted threshold may be different depending on the authorization result at the first gate 101A. Also, the same permitted threshold is used as the permitted threshold of the first gate 101A and the permitted threshold of the second gate 101B. However, a different permitted threshold may be used for each comparison depending on the image capture apparatus installation situation (outside light, angle of elevation of the installation, and the like) or the conditions desired by the user.
By restricting who can be an authorization target at the second gate to people with an authorization state of “authorized” or “authorized and deferred” or to people with an entry state of “first gate”, authorization can be performed quicker and with more accuracy than when performing authorization targeting all of the people registered in advance. However, in a case where the number of people registered in advance is low, to simplify processing, authorization may always be performed targeting all of the people. Also, in the example in
In a case where the processing arrives at step S709, the comparison result is set to “permitted”, and the notification control unit 305 notifies the person exiting that the comparison result is “permitted” via audio or lighting, as in step S609. In step S710, the state management unit 306 changes the entry state in the database managing the person to “first gate” as part of the exit permission processing. In step S711, the gate control unit 307 opens the second gate 101B and ends the processing in
In a case where the processing arrives at step S712, the comparison result is set to “not permitted”, and the notification control unit 305 notifies the person exiting that the comparison result is “not permitted” via audio or lighting. In step S713, the state management unit 306 executes special exit processing and ends the processing in
After the processing of step S809 or S813, the processing proceeds to step S810 where the state management unit 306 changes the authorization state in the database managing the person to “unauthorized” and the entry state to “has not entered” as part of the exit processing. In step S811, the gate control unit 307 opens the first gate 101A and ends the processing in
According to such a configuration, at the first-stage authorization gate, a person can be allowed to enter depending on whether or not a comparison similarity is equal to or greater than the first threshold, and at the second-stage authorization gate, a person can be allowed to enter depending on whether or not a comparison similarity is equal to or greater than the second threshold, which is greater than the first threshold. Accordingly, congestion can be prevented while maintaining a certain level of security accuracy in the first-stage authorization, and the normal level of security accuracy can be ensured in the second-stage authorization.
In the first embodiment, comparison is performed using a facial feature of a person registered in advance when comparing a person. In the present embodiment, the face when entering the first gate is additionally registered in the database and used thereafter in the comparison. The information processing apparatus 105 holds the (facial) feature of a person with a similarity equal to or greater than the first threshold upon entry at the first gate and performs a comparison at the second gate using the features added and held as the facial feature pre-registered in the database. Also, the information processing apparatus 105 changes the permitted threshold used for entry of the second gate depending on the first similarity calculated upon entry of the first gate. As described below, this can reduce the false rejection rate of registered people failing the authorization.
Also, even when the person entering is a person permitted to pass through and the person being referenced for comparison is the person in question, there may be some cases where, due to aging of the facial feature or the like, the similarity output as the comparison result is permanently reduced. From this perspective, the state management unit 306 according to the present embodiment can perform a determination of whether or not the first similarity upon authorization is less than a third threshold (update threshold) greater than the second threshold and, in a case where the first similarity is less than the third threshold, update the facial feature held in the database.
The overall configuration and arrangement of the apparatuses used to perform authorization using the information processing system according to the present embodiment and the hardware configuration and functional configuration of the information processing apparatus 105 are similar to those in the first embodiment. Thus, these will not be described here.
The authorization processing executed by the information processing apparatus 105 according to the present embodiment upon entry to the first gate 101A is basically similar to the processing executed in the first embodiment illustrated in
First, in step S901, the state management unit 306 changes the authorization state in the database managing the person to “authorized” and the entry state to “first gate”. This processing is similar to the processing in step S511 of the first embodiment.
In step S902, the state management unit 306 determines whether or not the first similarity when authorization is performed is less than the update threshold. In a case where the first similarity is less than the update threshold, the processing proceeds to step S903. Otherwise, the processing proceeds to step S905. Here, as the update threshold, an update threshold TH_OK+α is used, which is a threshold with a margin obtained by adding a predetermined margin value α to the permitted threshold (TH_OK).
In step S903, the notification control unit 305 notifies the person entering, prompting the person entering to update the registered facial feature. The notification control unit 305 may notify the person entering at that place via the notification apparatus 103 with lighting or audio and may notify the person entering using a portable terminal owned by the person entering or a (company) email address corresponding to the person entering. Also, the notification control unit 305 may notify the system administrator, prompting them to notify the person entering of a date and time to retake a face image with the system administrator.
In step S904, the state management unit 306 registers the temporary permitted threshold in the database. In the case of updating the registered facial feature, since the similarity is expected to be permanently lowered until an update is performed, as a temporary measure, the permitted threshold is temporarily lowered. The temporary permitted threshold, for example, may be a fixed value such as a threshold (TH_OK-β), which is a predetermined amount (β) less than the permitted threshold (TH_OK), or a dynamic value such as a threshold (SIM-β), which is a predetermined amount (β) less than the first similarity (SIM) when authorization is performed. The value of β can be set as desired by the user.
In the present embodiment, in step S502, a person is tracked, and in step S504, a plurality of facial features are extracted from a plurality of images of one person. However, in the determination of a comparison result in step S506, the similarity of the face which is the most similar is used. In step S905, the state management unit 306 checks whether or not there is a facial feature with a first similarity less than the permitted threshold (TH_OK) among the facial features in the plurality of images. In a case where there is a facial feature with a first similarity less than the permitted threshold, the processing proceeds to step S906. Otherwise, the entry permission processing in
Also, even when the person entering is a person permitted to pass through and the person being referenced for comparison is the person in question, if the face is facing sideward or downward or in a similar predetermined state not conducive for comparison, it is plausible that that the similarity obtained by calculation will be low. Also, even when the person is the person in question, if it is a predetermined state where, for example, the hair style or accessories such as glasses are different from those in the feature image registered in advance, it is plausible that the similarity obtained by calculation will be low. By registering the facial feature upon entry in such a predetermined state as a temporary facial feature and using it in the subsequent comparison when executing authorization processing, these daily changes in the person can be accommodated, and the false rejection rate can be reduced.
Note that in this example, a feature with a similarity that is less than the permitted threshold is registered as a temporary facial feature. However, a feature with a similarity that is equal to or greater than the permitted threshold may be registered as a temporary facial feature. Also, a face determined to be in a predetermined state, such as a face facing behind or straight down so that the face is not clearly seen, is not appropriate for facial authentication and is preferably not registered in as a temporary facial feature. A face in such a predetermined state may be excluded in step S905 or excluded when extracting a facial feature in step S504. Also, in the present embodiment, a temporary facial feature is registered. However, a facial feature may be registered (updated) as a non-temporary facial feature. In this manner, the information processing apparatus 105 may determine whether a feature in an image captured upon entry at the first gate is inappropriate to be used as a feature of a person, the subject, registered as a person permitted to pass through and may hold only images that are not inappropriate. The determination of whether it is inappropriate may be a determination of whether the face is not clearly seen as described above or a determination of whether or not a condition set by the user is satisfied.
As in step S905, in step S1002, the state management unit 306 checks whether or not there is a facial feature with a first similarity less than the permitted threshold (TH_OK) among the facial features in the plurality of images.
In a case where there is a facial feature with a first similarity less than the permitted threshold, the processing proceeds to step S1003. Otherwise, the entry hold processing in
The authorization processing executed by the information processing apparatus 105 according to the present embodiment upon entry at the second gate 101B is executed as in the flowchart illustrated in
The authorization unit 304 in step S605 according to the present embodiment performs a facial feature comparison of the temporarily registered facial feature with “authorized” as the authorization state in addition to the facial feature registered in the database in advance with “authorized” or “authorized and deferred” as the authorization state and obtains the registered face with the highest similarity.
In step S606, the authorization unit 304 determines whether or not the highest similarity corresponding to the face output in step S605 is greater than the second threshold (permitted threshold TH_OK). However, when a temporary permitted threshold is registered, the temporary permitted threshold is used instead of the permitted threshold. The entry permission processing is similar to the entry permission processing in
The flowchart illustrating an example of the authorization processing executed by the information processing apparatus 105 according to the present embodiment upon exit at the second gate 101B is similar to the flowchart in the first embodiment. However, in the exit permission processing of step S710, in addition to the exit permission processing of the first embodiment, a processing similar to the entry permission processing of
The authorization processing executed by the information processing apparatus 105 according to the present embodiment upon exit at the first gate 101A is executed as in the flowchart illustrated in
First, in step S1101, the state management unit 306 changes the authorization state in the database managing the person to “unauthorized” and the entry state to “has not entered”. This processing is similar to the processing in step S810 of the first embodiment. In step S1102, the state management unit 306 checks whether or not a facial feature update has been performed. In a case where it has been updated, the processing proceeds to step S1103. Otherwise, the processing proceeds to step S1104.
In step S1103, the state management unit 306 deletes the temporary permitted threshold from the database, and the processing proceeds to step S1104. In step S1104, the state management unit 306 deletes the temporary facial feature from the database, and the exit processing ends. Note that in the present embodiment, whether the facial feature has been updated is checked in step S1102, However, the temporary permitted threshold may be deleted each time.
According to such processing, the facial feature of a person extracted from an image upon entry is registered as a comparison target and used in comparison. Also, the permitted threshold may be changed via the similarity of a comparison upon entry. In this manner, the false rejection rate, where comparison fails even when it is the person in question, can be reduced.
In the first and second embodiment, at a first-stage authorization gate, 1:N authentication is performed to compare against a plurality of people registered in advance. The information processing apparatus 105 according to the present embodiment reads a medium, such as an employee ID card, held by the person entering and performs 1:1 authentication to compare a feature of the person registered in the medium and a feature of the person entering. In other words, a feature of a person seen in the camera and a facial feature associated with a storage medium storing ID information (identification information) owned by the person are compared. In the configuration according to the present embodiment, costs relating to presenting and reading ID information become necessary, but the accuracy of comparison is high due to a reduction in failures in which the wrong person is accepted. Also, compared to performing 1:N authentication, the processing time requiring for comparing is less due to only comparing whether or not the person is a specific person.
The storage medium according to the present embodiment may be a magnetic card, an ID card or smart device with a built-in IC chip, a card inscribed with a barcode or a QR code (registered trademark), or a smart device that can display such a code.
The information processing apparatus 105 according to the present embodiment reads the ID information from the storage medium via a contact or non-contact method using the reading apparatus illustrated in
In step S1508, the authorization unit 304 compares the facial feature of the person corresponding to the ID information and the facial feature extracted in step S1504 and obtains the registered face with the highest similarity. In this example, the facial feature of the person corresponding to the ID information is a facial feature registered in the database in advance that is obtained from the database. However, no such limitation is intended. For example, a read facial feature stored in the storage medium may be used as the facial feature of the person corresponding to the ID information.
In step S1509, the authorization unit 304 determines whether the highest similarity (the first similarity) corresponding to the face output in step S1508 is less than the first threshold (the not permitted threshold TH_NG), is equal to or greater than the second threshold (the permitted threshold TH_OK), or in between the two thresholds. When the similarity is equal to or greater than the permitted threshold, the processing proceeds to step S1510, when the similarity is less than the permitted threshold but equal to or greater than the not permitted threshold, the processing proceeds to step S1512, and when the similarity is less than the not permitted threshold, the processing proceeds to step S1515. The processing from steps S1510 to S1516 is similar to the processing from steps S510 to S516 in
The information processing apparatus 105 according to the present embodiment ensures a similar level of security at the second gate as at the first gate by the reading apparatus 1210B reading the ID information of a person with “authorized and deferred” as the authorization state.
The processing from steps S1601 to S1605 is similar to the processing from steps S601 to S605 in
In step S1607, the authorization unit 304 checks whether the authorization state of the person whose face is output in step S1605 is “authorized and deferred”. In a case where the authorization state is “authorized and deferred”, the processing proceeds to step S1608. Otherwise, the processing proceeds to step S1612.
In step S1608, the notification control unit 305 notifies the person via audio or a display on a monitor, prompting them to present the storage medium storing the ID information owned by the person to the reading apparatus 1210B. In step S1609, the ID reading unit 1408 reads the ID information from the storage medium presented by the person via the reading apparatus 1210B.
In step S1610, the authorization unit 304 compares the facial feature of the person corresponding to the ID information and the facial feature extracted in step S1604 and obtains the registered face with the highest similarity. In step S1611, the authorization unit 304 determines whether or not the similarity determined to be the highest in step S1610 is greater than the second threshold (permitted threshold TH_OK). When the similarity is equal to or greater than the permitted threshold, the processing proceeds to step S1612. In a case where the similarity is less than the permitted threshold, the processing proceeds to step S1617.
In step S1612, the authorization unit 304 checks whether or not the person whose registered face is obtained in step S1610 has the authority to pass through the second gate 101B. In a case where the person has passage authority, the processing proceeds to step S1614. In a case where the person does not have passage authority, the processing proceeds to step S1617. The processing from steps S1614 to S1618 is similar to the processing from steps S609 to S613 in
In this manner, in an entry and exit management system installed with multi-stage authorization gates, the information processing apparatus 105 can perform a comparison of a facial feature associated with the storage medium storing ID information owned by a person at a first-stage authorization gate. Also, at a second-stage authorization gate, the information processing apparatus 105 can only perform a comparison of the facial feature associated with the storage medium again for a person determined as “deferred” at the first-stage authorization gate and compare the facial feature for a person determined as “permitted” at the first-stage authorization gate. According to such processing, congestion can be prevented while maintaining a certain level of security accuracy at the first-stage authorization gate, and the normal level of security accuracy can be ensured in the second-stage authorization gate. Note that in a derived embodiment for increasing the security, at the second-stage authorization gate, at all times, a comparison of the facial feature associated with the storage medium storing the ID information may be performed.
In the third embodiment, an ID reading apparatus is installed at the second-stage authorization gate. To improve the user-friendliness, in the present embodiment, second-stage authorization gates (for entering the building) are installed at a plurality of locations.
Here, one or more of the authorization gates may be configured without an ID reading apparatus to reduce installation costs.
The processing from steps S1801 to S1807 is similar to the processing from steps S1601 to S1607 in
Note that to prompt action to head directly to the second gate 101B and not the second gate B(101C) where no reading apparatus is installed, the information processing apparatus 105 can notify the person entering that their result when entering the first gate 101A is “deferred”. Also, a place other than a gate where an ID reading apparatus can be installed, and people with the authorization state of “authorized and deferred” can be prompted to be authorized at this place to obtain the authorized state.
According to such a configuration, authorization processing can be executed without installing an ID reading apparatus at all of the authorization gates. Thus, the installation cost can be reduced while maintaining security accuracy.
In the first to fourth embodiments, two-stage authorization gates are prepared and processing is executed. However, the number of authorization gates is not particularly limited to two. In the present embodiment, three-stage or more authorization gates are used to perform a comparison of a person entering.
As described above, the information processing apparatus 105 according to the present embodiment checks whether the person entering imaged by the image capture apparatus 102D at the third gate 101D is a person registered as a person permitted to pass through the third gate 101D and categorizes the authorization results. In a case where three-stage authorization gates are used as illustrated in
The information processing apparatus 105 according to the present embodiment calculates the similarity (a third similarity) between the feature of a person imaged by the image capture apparatus 102D and the feature of a register person and determines whether or not the calculated similarity is less than a fourth threshold. Next, in a case where the third similarity is less than the fourth threshold, the information processing apparatus 105 performs control to not permit passage at the third gate 101D. In this example, the fourth threshold is the same value as the second threshold, but the value may be set as desired by the user depending on desired conditions. For example, the value may be set to a value greater than the second threshold.
In a similar manner in a different embodiment, the third gate 101D may be installed between the first gate 101A and the second gate 101B (in other words, installed as a second-stage gate) (in this case, the second gate 101B is referred to as a third-stage gate). In this case, the information processing apparatus 105 may categorize the authorization results into “permitted”, “deferred”, and “not permitted” for the first-stage and second-stage authorization gate and “permitted” and “not permitted” for the third-stage authorization gate.
In this manner, the information processing apparatus 105 can determine whether the calculated third similarity is less than a fifth threshold less than the fourth threshold, equal to or greater than the fifth threshold but less than the fourth threshold, or is equal to or greater than the fourth threshold. In this case, the information processing apparatus 105 classifies the authorization result as permitted, deferred, or not permitted on the basis of whether the third similarity is equal to or greater than the fourth threshold, less than the fourth threshold but equal to or greater than the fifth threshold, or less than the fifth threshold. In this example, the fifth threshold is the same value as the first threshold, but the value may be set as desired by the user depending on desired conditions. For example, the value may be set to a value greater than the first threshold. Such a configuration is particularly suitable in a case where the place between the second-stage authorization gate and the third-stage authorization gate is a place with a relatively low security clearance such as a cafeteria and many people need to passing through the second-stage authorization gate.
Note that in the example in
In the first to fifth embodiment, authorization processing is executed by giving permission to pass through to a person registered in a database in advance. However, it is plausible that there may be some cases in which a visitor or the like that is not registered in the database in advance is given permission to pass through. However, normally, when a visitor not registered in the database in advance tries to enter from the first-stage gate, entrance is not permitted due to the similarity of the comparison result being less than the permitted threshold.
From this perspective, the information processing apparatus 105 according to the present embodiment notifies the person entering when the first similarity is less than the first threshold (when the authorization result is “not permitted”). Here, for example, the information processing apparatus 105 may notify the person entering with an instruction guiding them to a reception where a non-illustrated image capture apparatus is installed to register a temporary facial feature by a face image of the visitor being captured, for example. This guidance instruction may be normally displayed near the gate for visitors. In a case where this notification is issued, an image is captured of the visitor at the reception, and a facial feature in the captured image is registered in the database as a temporary facial feature, the visitor can be given “permitted” as the authorization state by executing authorization processing similar to the authorization processing in the first embodiment using a temporary facial feature. Also, in a case where a facial feature in a face image of the visitor is registered as a temporary facial feature at the reception, at this point in time, the authorization state for the first gate may be set to “permitted” and the person may be permitted to pass through the first gate.
In the embodiments described above, three categories of “permitted”, “deferred”, and “not permitted” are used at the first-stage authorization gate. However, “permitted” and “deferred” maybe further categorized into multiple levels. For example, the deferred category may be further split into two level using an additional threshold, with different predetermined control being performed. For example, with one, 1:1 authentication using a reading apparatus at the second gate may be executed, and with the other, a threshold may be changed. Also, the information processing apparatus 105 may store a value for the similarity calculated at the first gate and execute authorization processing using a threshold set per gate (depending on the security clearance) on the basis of the similarity stored without imaging the person entering there and then when they pass through the gate.
Also, in the second embodiment, in a case where the permitted threshold is exceeded by a significant margin, a notification may be issued prompting the person to update the facial feature. However, the condition for issuing such a notification is not particularly limited thereto, and a notification prompting the person to update the facial feature may be issued depending on the authorization processing history, for example. For example, in a case where the authorization state for a person is determined as “deferred” at an increasing frequency (for example, equal to or greater than a predetermined ratio) over a predetermined period (or a predetermined number of authorizations), the notification control unit 305 may issue a notification prompting the person to update the facial feature.
According to the information processing system and the information processing method of the embodiments described above, in a case where multi-stage authorization places (for example, authorization gates) are provided, the security accuracy can be ensured while preventing congestion. Specifically, congestion can be prevented while maintaining a certain level of security accuracy in the first-stage authorization, and the normal level of security accuracy can be ensured in the second-stage authorization.
The present invention is not limited to the above embodiments and various changes and modifications can be made within the spirit and scope of the present invention. Therefore, to apprise the public of the scope of the present invention, the following claims are made.
Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)TM), a flash memory device, a memory card, and the like.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2023-008949, filed Jan. 24, 2023, which is hereby incorporated by reference herein in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2023-008949 | Jan 2023 | JP | national |
