Patent Application
20240312213
The described aspects relate to security systems that comprise object detection features.
Aspects of the present disclosure relate generally to vision systems that may be used for security, and more particularly to object detection.
Vision systems may be used to detect objects in an environment. In some implementations, for example, vision systems may include object detection and counting capabilities. For instance, in some cases, the object may be a person, and the object counting may be used for determining occupancy counts, which is important for fields such as security, marketing, and health. For example, a user of a security system may be interested in knowing how many people have entered/exited a theater with a fire code occupancy limit. An owner of a shopping mall may be interested in knowing how many people enter different stores to evaluate popularity. Office personnel may be interested in knowing how many people have entered an office to enforce health-based regulations (e.g., limit the occupancy count to prevent the spread of a virus).
Conventional systems, however, often fail to provide accurate detection and/or counts due to sub-par detection algorithms, aging/corrupted hardware, and different exit/entry scenarios (e.g., visual/physical obstacles, unorthodox entry/exit movement by people, etc.). Accordingly, there exists a need for improvements in such vision systems with objection detection/counting capabilities.
The following presents a simplified summary of one or more aspects in order to provide a basic understanding of such aspects. This summary is not an extensive overview of all contemplated aspects, and is intended to neither identify key or critical elements of all aspects nor delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.
An example implementation includes a method for use by a vision system for detecting objects in an environment, comprising detecting, using at least one sensor, persons that entered and exited the environment during a first period of time. The method further includes determining an entry count and an exit count for the first period of time. The method further includes retrieving, from a database, historical ingress and egress data comprising entry and exit counts of the environment for a second period of time. The method further includes determining an amount of expected exit counts for the first period of time based on the historical ingress and egress data of the second period of time by fitting the historical ingress and egress data to a first probability distribution. The method further includes updating the exit count for the first period of time to the amount of expected exit counts. The method further includes storing the updated exit count in the database.
Another example implementation includes a vision system for detecting objects in an environment, comprising a memory and a processor in communication with the memory. The processor is configured to detect, using at least one sensor, persons that entered and exited the environment during a first period of time. The processor is configured to determine an entry count and an exit count for the first period of time. The processor is configured to retrieve, from a database, historical ingress and egress data comprising entry and exit counts of the environment for a second period of time. The processor is configured to determine an amount of expected exit counts for the first period of time based on the historical ingress and egress data of the second period of time by fitting the historical ingress and egress data to a first probability distribution. The processor is configured to update the exit count for the first period of time to the amount of expected exit counts and store the updated exit count in the database
Another example implementation includes a vision system for detecting objects in an environment, comprising means for detecting, using at least one sensor, persons that entered and exited the environment during a first period of time. The apparatus further includes means for determining an entry count and an exit count for the first period of time. The apparatus further includes means for retrieving, from a database, historical ingress and egress data comprising entry and exit counts of the environment for a second period of time. The apparatus further includes means for determining an amount of expected exit counts for the first period of time based on the historical ingress and egress data of the second period of time by fitting the historical ingress and egress data to a first probability distribution. The apparatus further includes means for updating the exit count for the first period of time to the amount of expected exit counts, and storing the updated exit count in the database.
Another example implementation includes a computer-readable medium storing instructions, for use by a vision system for detecting objects in an environment, executable by a processor to detect, using the at least one sensor, persons that entered and exited the environment during a first period of time. The instructions are further executable to determine an entry count and an exit count for the first period of time. The instructions are further executable to retrieve, from a database, historical ingress and egress data comprising entry and exit counts of the environment for a second period of time. The instructions are further executable to determine an amount of expected exit counts for the first period of time based on the historical ingress and egress data of the second period of time by fitting the historical ingress and egress data to a first probability distribution. The instructions are further executable to update the exit count for the first period of time to the amount of expected exit counts and store the updated exit count in the database.
To the accomplishment of the foregoing and related ends, the one or more aspects comprise the features hereinafter fully described and particularly pointed out in the claims. The following description and the annexed drawings set forth in detail certain illustrative features of the one or more aspects. These features are indicative, however, of but a few of the various ways in which the principles of various aspects may be employed, and this description is intended to include all such aspects and their equivalents.
The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate one or more example aspects of the present disclosure and, together with the detailed description, serve to explain their principles and implementations.
Various aspects are now described with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It may be evident, however, that such aspect(s) may be practiced without these specific details.
The present disclosure describes implementations of improving vision systems that include occupancy counting features. Vision systems often fail to provide accurate counts of objects due to object detection issues. To address these shortcomings, the present disclosure describes how to improve occupancy counts using historical object detection data.
Suppose that person 104 enters the environment. The occupancy counting system of the present disclosure may use a facial detection algorithm to identify person 104 at a certain position in a captured frame (e.g., frame 100). Over the course of multiple frames, the occupancy counting system may continue to detect person 104 and monitor the changes in position of person 104. Based on the facial features of person 104, the occupancy counting system may determine that the person in one frame is the same as the person in the next frame. In terms of tracking, a point on the face of person 104 may be located at position (x1, y1) in a first frame and may be located at position (x2, y2) in a second frame. The occupancy count system may determine that the difference between positions of a plurality of points where person 104 is tracked (including points (x1, y1) and (x2, y2)) is indicative of a trajectory of person 104. For example, a trajectory away from access point 102 may indicate an entry. A trajectory towards access point 102 may indicate an exit.
When person 104 enters the environment, the occupancy count may rise from 3 (including person 106 and persons 108) to 4. Suppose that person 106 then leaves the environment. The occupancy count may decrease back to 3. As previously explained, certain entries and exits may be missed. For example, persons 108 may exit, which should account for two exits. However, the camera may not detect two people because of their close proximity (e.g., one person may block the view of the camera as persons 108 exit). The occupancy count correction (OCC) component of an occupancy counting system described in
In some aspects, with this corrected information, the occupancy counting system may execute different commands depending on a goal set by an administrator. For example, if the administrator is seeking to limit the amount of entries to a maximum occupancy count, when the corrected occupancy count reaches the maximum occupancy count, the system may transmit an alert (e.g., generate and send an email or text message via a wired or wireless network) to the administrator or automatically prevent further entries (e.g., disable access point 102 access by activating a lock or deactivating entry motion detection if access point 102 are motion-detection-based). In other aspects, the administrator may seek to determine the amount of time people spend in the environment to determine an open/close schedule (e.g., close the store at 8:00 pm instead of 7:00 pm due to the high entry traffic at 6:45 pm).
The occupancy count at a given time is given by the difference in entry counts and exit counts. For example, if in a given time period five people enter and two people exit an environment, the occupancy count is the previous occupancy count plus the difference (i.e., the net entry/exit count). If the occupancy count was previously 0, the new occupancy count is three. Ideally, both the entry and exit counts are accurate. If even one of them is incorrect, the occupancy count is also incorrect.
As discussed previously, there are many reasons why the entry and exit counts may be incorrect. Occupancy counting solutions without a means for error correction will suffer from accumulated accuracy loss over time. The presented idea of real-time occupancy counting corrections using the described approach is unique. Observed typical prior approaches either utilize correlation with hardware devices or do not provide a real-time and active learning approach proposed by the present disclosure.
The use of statistical methods identifies the error in an occupancy or foot-traffic system's exit/entry count and corrects it in real-time. This probabilistic approach addresses the sensor noise/corrupted data challenge and potentially identifies malfunctioning devices. The proposed OCC component learns the distribution of the data, provides parameters, and as new data comes in, the model automatically corrects the system's exit/entry count—offering higher accuracy and solving the non-detection/undercounting problem.
In terms of exits, in some aspects, the beta-binomial regression algorithm is utilized by the OCC component. The beta distribution is a probability distribution that represents all the possible values of a probability when one does not know what that probability is. Consider a Missed Exit Percentage (MEP) as the predictor, given exit counts and entries counts. MEP is the percentage of missed exits (percentage between 0 and 1). The beta distribution is a perfect fit for this problem since it lies within this domain. The OCC component estimates separate priors for a given period of time (e.g., each day of the week). One useful approach to this is Bayesian hierarchical modeling.
In the event that the period of time is large (e.g., giving more consideration to the days with a higher percentage of missed exits), the OCC component utilizes the beta-binomial distribution. It should be noted that as more entries are detected, the higher the probability of missed exits (i.e., MEP). Thus, there is a relationship between the entries count and MEP.
In some aspects, each period of time (e.g., each day of the week) has its own prior associated with the beta-binomial regression model. This establishes a general framework for allowing a prior to depend on known information. Accordingly, the OCC component estimates hyperparameters a0 (related to success) and B0 (related to failures) for empirical Bayes. For each day, the OCC component updates the beta prior based on the evidence to get posterior parameters a1 and B1. The OCC component generates a data structure of historical ingress and egress data indicating date, entry counts, exit counts, MEP, an empirical Bayes Estimate that lists an estimate about MEP for each time unit (e.g., day) in the period of time (estimated from a combination of each time unit's record with the beta prior parameters estimated across all time units (a0, B0)), a1, and B1.
To avoid under estimation in exit counts the OCC component accounts for entries in the model (as entries influence priors and particularly affect the MEP). In particular, the typical MEP is to be linearly affected by log(Entries). Defining pi to be the true probability of exits for a time unit (i.e., “true MEP”), the OCC component runs based on the following:
The OCC component first fits these prior parameters: μ0, μEntries, σ0. The OCC component momentarily “forgets” Bayes and picks a0 and B0 using a maximum likelihood. The OCC component then fits beta-binomial regression using the maximum likelihood to obtain μ0, μEntries, σ0 (establishing the dependency of the prior beta distribution for a time unit on the value of the entry count). The OCC component then repeats the second step of the empirical Bayes method. Instead of using a single a0 and B0 values as the prior, however, the OCC component chooses the prior for each time unit (e.g., day) based on their entry counts. For example, there may be different a0 and B0 values for Monday than there may be for Tuesday. The OCC component then updates using their E and Entries and calculates μ=μ0+μlog(Entries) and σ parameters for each person. In some aspects, σ will be the same for every time unit (but that may not be true in more complex models).
The OCC component then calculates a0 and B0 parameters for each time unit, according to a0,i=μ0/σ0 and B0,i=(1−μi)/σ0. From this, the OCC component updates E and Entries to calculate the new a1,i and B1,i for each time unit. Lastly, the OCC component calculates an exit ratio based on the time unit in a period of time (e.g., a day in a week), and multiplies the exit ratio with the detected exit count to get a corrected exit count. For example, if the detected exit count is 10 and the exit ratio is 1.2, the corrected exit count becomes 12.
Thus far, the focus has been on counting exits correctly. In terms of counting entries correctly, a different approach is used (because entries are mostly independent of prior exits and other entries). One issue with entry detection is that occupancy counting systems detect multiple “zero” entries. Consider the following dataset:
As can be seen in the table, at 23:00, there are zero entries. This seems like a realistic count because the environment may be a store that is closed at that time. However, the table also indicates that at noon, there were zero entries. This needs correction. Suppose that the table has a total of 3024 entries and 412 of those entries indicate zero as the entry count. Of these zeroes, only 10 are true zeroes (e.g., zeroes where the store was closed). The OCC component utilizes a probability mass function of the Zero-Inflated Poisson model to determine the correct entry count.
First, the OCC component calculates φi using logistic regression as the ratio of zeros for the ith row in the dataset above. For example, of the 412 zero entries, 10 are true zeroes-thus 402 are potentially not true zeroes. This gives a φi value of 402/3024=0.133.
When the entry count is zero, the OCC component calculates P(yi=0)=φi+(1−φi)*e−λi and when the entry count is not zero, the OCC component calculates
where λi=ex
It should be noted that when estimating either exit counts or entry counts, detection models have more errors when larger groups of people are present. In some aspects, the algorithms executed by OCC component may be adjusted to account for group size. For example, the OCC component may use the correction algorithm involving the probability distributions particularly for time periods where the group size is greater than a threshold size. In another example, the OCC component may detect image frames where more than a threshold number of people are detected. The OCC component may store the time period associated with the image frames and run the correction algorithm for the stored time period.
At block 402, the method 400 includes detecting, using at least one sensor, persons that entered and exited the environment during a first period of time. For example, in an aspect, computer device 300, processor 305, memory 310, OCC component 315, and/or detecting component 320 may be configured to or may comprise means for detecting, using the camera that captured the image in
For example, detecting component 320 may use facial recognition and tracking algorithms to detect and monitor persons 104, 106, and 108 over a plurality of image frames.
At block 404, the method 400 includes determining an entry count and an exit count for the first period of time. For example, in an aspect, computer device 300, processor 305, memory 310, OCC component 315, and/or determining component 325 may be configured to or may comprise means for determining an entry count (e.g., 4) and an exit count (e.g., 2) for the first period of time.
For example, in
At block 406, the method 400 includes retrieving, from a database, historical ingress and egress data comprising entry and exit counts of the environment for a second period of time. For example, in an aspect, computer device 300, processor 305, memory 310, OCC component 315, and/or retrieving component 330 may be configured to or may comprise means for retrieving, from a database (e.g., in memory 310), historical ingress and egress data comprising entry and exit counts of the environment for a second period of time.
In some aspects, OCC component 315 may generate a data structure of historical ingress and egress data indicating date, entry counts, exit counts, MEP, an empirical Bayes Estimate that lists an estimate about MEP for each time unit (e.g., day) in the period of time (estimated from a combination of each time unit's record with the beta prior parameters estimated across all time units (a0, B0)), a1, and B1. In particular, OCC component 315 updates the beta prior based on the evidence to get posterior parameters a1 and B1.
In some aspects, the first period of time and the second period of time are at a same time of day across different days. For example, the second period of time may be Sunday between 10:00 am and 12:00 pm. In some aspects, the different days are a same day across different weeks. For example, the second period of time may be Monday from the previous week between 10:00 am and 12:00 pm.
At block 408, the method 400 includes determining an amount of expected exit counts for the first period of time based on the historical ingress and egress data of the second period of time by fitting the historical ingress and egress data to a first probability distribution. For example, in an aspect, computer device 300, processor 305, memory 310, OCC component 315, and/or determining component 335 may be configured to or may comprise means for determining an amount of expected exit counts for the first period of time (e.g., 12 exits) based on the historical ingress and egress data of the second period of time by fitting the historical ingress and egress data to a first probability distribution. In some aspects, the first probability distribution is a Beta-Binomial distribution.
In an exemplary aspect, OCC component 315 accounts for entries in the model (as entries influence priors and particularly affect the MEP). In particular, the typical MEP is to be linearly affected by log(Entries). Defining pi to be the true probability of exits for a time unit (i.e., “true MEP”), OCC 315 component runs based on the following:
OCC component 315 estimates hyperparameters a0 and B0 for empirical Bayes using a maximum likelihood. OCC component 315 then fits beta-binomial regression using the maximum likelihood to obtain μ0, μEntries, σ0. OCC component 315 then repeats the second step of the empirical Bayes method. OCC component 315 then updates using the E and Entries values and calculates u=μ0+μlog(Entries) and σ parameters for each person.
OCC component 315 then calculates a0 and B0 parameters for each time unit, according to a0,i=μi/σ0 and B0,i=(1−μi)/σ0. From this, OCC component 315 updates E and Entries to calculate the new a1,i and B1,i for each time unit. Lastly, OCC component 315 calculates an exit ratio based on the time unit in a period of time (e.g., a day in a week), and multiplies the exit ratio with the detected exit count to get a corrected exit count. For example, if the detected exit count is 10 and the exit ratio is 1.2, the corrected exit count becomes 12.
An example of exit miss probabilities is shown in the table below:
At block 410, the method 400 includes updating the exit count for the first period of time to the amount of expected exit counts. For example, in an aspect, computer device 300, processor 305, memory 310, OCC component 315, and/or updating component 340 may be configured to or may comprise means for updating the exit count for the first period of time to the amount of expected exit counts. For example, the exit count for the first period of time may be 10 and the amount of expected exit counts may be 12.
At block 412, the method 400 includes storing the updated exit count in the database. For example, in an aspect, computer device 300, processor 305, memory 310, OCC component 315, and/or storing component 345 may be configured to or may comprise means for storing the updated exit count in the database (e.g., in memory 310). For example, instead of recording 10 exits, the updated count of 12 exits is recorded in the database. In some aspects, the stored count is included in the historical ingress and egress data.
At block 502, the method 400 includes determining an amount of expected entry counts for the first period of time by fitting historical ingress data of the second period of time to a second probability distribution. For example, in an aspect, computer device 300, processor 305, memory 310, OCC component 315, and/or determining component 350 may be configured to or may comprise means for determining an amount of expected entry counts for the first period of time by fitting historical ingress data of the second period of time to a second probability distribution (e.g., the Poisson distribution).
Referring again to table 1, which may represent historical ingress data of the second period of time, OCC component 315 may calculate φi using logistic regression as the ratio of zeros for the ith row in table 1. For example, of 412 zero entries, 10 are true zeroes, and 402 are potentially not true zeroes. This gives a φi value of 402/3024=0.133. Suppose that the detected entry count for the first period of time is 20. In this case, OCC component 315 may calculate
where λi=ex
After performing the calculation, the amount of expected entry counts may be 25 instead of 20.
At block 504, the method 400 includes updating the entry count for the first period of time to the amount of expected entry counts, wherein the updated entry count is used to determine the amount of expected exit counts. For example, in an aspect, computer device 300, processor 305, memory 310, OCC component 315, and/or updating component 340 may be configured to or may comprise means for updating the entry count for the first period of time to the amount of expected entry counts.
As mentioned above, the amount of expected exit counts may be used to calculate the updated entry count. For example, suppose that the detected entry count is 20 and the detected exit count is 10 for the first period of time. After the entry count is updated to 25, the number of exit counts is also likely to change. Referring to the calculation in
At block 506, the method 400 includes storing the updated entry count in the database. For example, in an aspect, computer device 300, processor 305, memory 310, OCC component 315, and/or storing component 345 may be configured to or may comprise means for storing the updated entry count in the database.
At block 602, the method 400 includes determining whether the entry count is zero. For example, in an aspect, computer device 300, processor 305, memory 310, OCC component 315, and/or determining component 325 may be configured to or may comprise means for determining whether the entry count is zero.
If the entry count is not equal to zero, method 600 advances to 604. Otherwise, method 600 advances to 606.
At block 604, the method 400 includes determining the amount of expected entry counts using a Poisson distribution (as explained in
At block 606, the method 400 includes calculating a probability of the entry count being zero using logistical regression on the historical ingress data. For example, in an aspect, computer device 300, processor 305, memory 310, OCC component 315, and/or calculating component 351 may be configured to or may comprise means for calculating a probability of the entry count being zero using logistical regression on the historical ingress data.
At block 608, the method 400 includes determining the amount of expected entry counts using a Zero-Inflated Poisson (ZIP) distribution. For example, in an aspect, computer device 300, processor 305, memory 310, OCC component 315, and/or determining component 350 may be configured to or may comprise means for determining the amount of expected entry counts using a Zero-Inflated Poisson (ZIP) distribution.
For example, when the entry count is zero, OCC component 315 calculates P(yi=0)=φi+(1−φi)*eλi, where λi=ex
While the foregoing disclosure discusses illustrative aspects and/or embodiments, it should be noted that various changes and modifications could be made herein without departing from the scope of the described aspects and/or embodiments as defined by the appended claims. Furthermore, although elements of the described aspects and/or embodiments may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated. Additionally, all or a portion of any aspect and/or embodiment may be utilized with all or a portion of any other aspect and/or embodiment, unless stated otherwise.
The application claims the benefit of U.S. Provisional Application No. 63/217,483, filed Jul. 1, 2021, which is herein incorporated by reference.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/US2022/073338 | 7/1/2022 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 63217483 | Jul 2021 | US |
