Patent Application
20230143934
The present disclosure pertains generally to mobile security devices that carry video cameras and more particularly to methods and systems for performing video analytics on video captured by such mobile security devices.
Mobile devices such as but not limited to drones, smart phones, tablets, and other mobile devices, often carry video cameras that can capture video of a scene. Some mobile devices include a controller that can perform some level of video analytics on the captured video. Mobile devices can often travel in a variety of different environments. For example, a drone can fly in both indoor and outdoor environments, and depending on the current task assigned to the drone, the drone may be looking for different types of security incidences. A need remains for improved methods and systems for performing suitable video analytics on video captured by mobile security devices.
This disclosure generally to mobile security devices that carry video cameras and more particularly to mobile security devices that carry video cameras and perform video analytics, where the video analytics that are performed are based at least in part on a location of the mobile security device when the video was captured. An example is found in a drone. An illustrative drone includes a video camera that is carried by the drone, a memory that is configured to store a plurality of video analytics algorithms, a position sensor, a transceiver and a controller that is operably coupled to the video camera, the memory, the position sensor and the transceiver. The controller is configured to determine a position of the drone based on information provided by the position sensor and to select one or more video analytics algorithms of the plurality of video analytics algorithms based at least in part upon the determined position of the drone. The controller is configured to instruct the video camera to capture video and to perform the selected one or more video analytics algorithms on the captured video, resulting in one or more video analytics results. In some cases, the controller is configured to transmit one or more of the video analytics results to a remote device via the transceiver.
Another example is found in a surveillance system that includes a mobile security device. The mobile security device includes a video camera that is carried by the mobile security device, a memory that is configured to store a plurality of video analytics algorithms, a position sensor, a transceiver and a controller that is operably coupled to the video camera, the memory, the position sensor and the transceiver. The controller is configured to instruct the video camera to capture video and save the captured video to the memory. The controller is configured to determine a position of the mobile security device, based on information provided by the position sensor, at a time that is representative of when the video camera captured the video, and to transmit at least part of the captured video saved in the memory along with the corresponding position.
In some instances, the surveillance system also includes a remote device. The remote device includes a memory that is configured to store a plurality of video analytics algorithms, a transceiver, and a controller that is operably coupled to the memory and the transceiver of the remote device. In some cases, the controller of the remote device is configured to receive the at least part of the captured video and the corresponding position transmitted by the mobile security device and to select one or more video analytics algorithms of the plurality of video analytics algorithms based at least in part upon the received position of the mobile security device. In some cases, the controller of the remote device is configured to perform the selected one or more video analytics algorithms on the received captured video, resulting in one or more video analytics results, and to store the one or more video analytics results.
Another example is found in a method of performing video surveillance. A video is captured from a location, and a location indicator representative of the location is stored. One or more video analytics algorithms of a plurality of video analytics algorithms are selected based at least in part on the location indicator. The selected one or more video analytics algorithms are performed on the captured video, resulting in one or more results. At least some of the one or more results are displayed on a display.
The preceding summary is provided to facilitate an understanding of some of the features of the present disclosure and is not intended to be a full description. A full appreciation of the disclosure can be gained by taking the entire specification, claims, drawings, and abstract as a whole.
The disclosure may be more completely understood in consideration of the following description of various illustrative embodiments of the disclosure in connection with the accompanying drawings, in which:
While the disclosure is amenable to various modifications and alternative forms, specifics thereof have been shown by way of example in the drawings and will be described in detail. It should be understood, however, that the intention is not to limit aspects of the disclosure to the particular illustrative embodiments described. On the contrary, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the disclosure.
The following description should be read with reference to the drawings wherein like reference numerals indicate like elements. The drawings, which are not necessarily to scale, are not intended to limit the scope of the disclosure. In some of the figures, elements not believed necessary to an understanding of relationships among illustrated components may have been omitted for clarity.
All numbers are herein assumed to be modified by the term “about”, unless the content clearly dictates otherwise. The recitation of numerical ranges by endpoints includes all numbers subsumed within that range (e.g., 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.80, 4, and 5).
As used in this specification and the appended claims, the singular forms “a”, “an”, and “the” include the plural referents unless the content clearly dictates otherwise. As used in this specification and the appended claims, the term “or” is generally employed in its sense including “and/or” unless the content clearly dictates otherwise.
It is noted that references in the specification to “an embodiment”, “some embodiments”, “other embodiments”, etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is contemplated that the feature, structure, or characteristic may be applied to other embodiments whether or not explicitly described unless clearly stated to the contrary.
Mobile security devices can be used for surveillance in a variety of different circumstances, both indoors and outdoors. While drones are used as a primary example, the present disclosure is applicable to other forms of mobile security devices including smart phones, tablets and/or any other suitable mobile device with a video camera. For indoors use, drones can be used in any large facility to provide a video camera anywhere that a video camera is desired. Drones can have standard patrol patterns, for example, although if there is an incident or suspected incident within a particular location that does not have fixed video cameras, or has limited video camera coverage, additional drones can be instructed to proceed to the particular location in order to provide additional video coverage of the incident. The drones may be instructed to proceed to the particular location by a centralized security system controller, for example, or the drones may communicate with each other and may independently decide where they should fly.
Indoor facilities that can accommodate drones include but are not limited to large facilities such as airport terminals, airplane hangers, manufacturing facilities, parking garages, shopping malls, sporting facilities and the like. For outdoor use, drones can be used to patrol or provide additional video capability to parking lots and traffic surveillance for roadways, amusement parks, and the like. Drones can also be used outdoors in a temporary fashion, to provide video surveillance capability for temporary events such as parades, demonstrations and protests, for example.
The illustrative mobile security device 10 includes a position sensor 18, a transceiver 20 and a controller 22 that is operably coupled to the video camera 12, the memory 14, the position sensor 18 and the transceiver 20. The controller 22 is configured to determine a position of the mobile security device 10 based on information provided by the position sensor 18 and to select one or more video analytics algorithms of the plurality of video analytics algorithms 16 based at least in part upon the determined position of the mobile security device 10.
In some cases, when a single video analytics algorithm is selected based on the position of the mobile security device 10, Table 1 below provide an example correlation between a position or region in a monitored area and a corresponding video analytics algorithm. In Table 1, each position 1, 2, 3, 4, and 5 corresponds to predefined set of coordinates ((x, y, z), (x+δx, y+δy, z+δz)) in the monitored area. When a video is captured by the mobile security device 10 when the position of the mobile security device 10 falls within one of these sets of coordinates, the corresponding video analytics algorithm is selected and applied to the captured video.
In Table 1, when the mobile security device 10 falls within the predefined set of coordinates that correspond to position 1, a people count video analytics algorithm is selected. When the mobile security device 10 falls within the predefined set of coordinates that correspond to position 2, a loitering detection video analytics algorithm is selected. When the mobile security device 10 falls within the predefined set of coordinates that correspond to position 3, an intrusion detection video analytics algorithm is selected. When the mobile security device 10 falls within the predefined set of coordinates that correspond to position 4, an occupancy detection video analytics algorithm is selected. When the mobile security device 10 falls within the predefined set of coordinates that correspond to position 5, vehicle count video analytics algorithm is selected.
In some cases, there may be a desire to perform two or more different video analytics algorithms, depending on the position of the mobile security device 10. Table 2 below provides an example of various positions for which performing two or more different video analytics algorithms is desired.
In some cases, selecting one or more video analytics algorithms may include selecting a single video analytics algorithm out of the plurality of video analytics algorithms 16. In some instances, selecting one or more video analytics algorithms may include selecting two or more different video analytics algorithms out of the plurality of video analytics algorithms 16. In the example shown in
When the controller 22 performs some video analytics algorithms, the controller 22 is configured to transmit one or more of the video analytics results to remote device 24 via the transceiver 20. In some cases, the remote device 24 may be a desktop computer or a cloud-based server. The remote device 24 may be part of a surveillance system control device, for example.
In some cases, the position sensor 18 may be configured to enable the controller 22 to ascertain an indoor position via triangulation. For example, the position sensor 18 may be configured to be able to triangulate between multiple beacons that are disposed within an indoors facility. The position sensor 18 may be configured to use a 5G cellular network to ascertain its position (either indoors or outdoors). In some cases, the position sensor 18 may be configured to triangulate its position using a magnetometer, ultrawide band (UWB) or even BLE (Bluetooth low energy). The position sensor 18 may have access to a floorplan of a facility by communicating with a BIM (building information model) or even a BMS (building management system). For outdoor locations, GPS (global positioning system), triangularization and/or other suitable technique may be used to ascertain position.
In this, position may refer strictly to the physical location of the mobile security device 10. For example, the position of the mobile security device 10 is 5 meters due south of door #14 within the facility. In some cases, position may also take into account the field of view of the video camera 12 of the mobile security device 10. While the mobile security device 10 may currently be 5 meters due south of door #14 within the facility, the video camera 12 may have a field of view that extends westward down hallway #4. In some cases, position can be a combination of physical location and field of view.
The mobile security device 32, which may be configured to fly, includes a video camera 36, a position sensor 38, a memory 40, a transceiver 42 and a controller 44 that is operably coupled to the video camera 36, the position sensor 38, the memory 40 and the transceiver 42. The controller 44 is configured to instruct the video camera 36 to capture video and save the captured video to the memory 40. The controller 44 is configured to determine a position of the mobile security device 32, based on information provided by the position sensor 38, at a time that is representative of when the video camera 36 captured the video. The controller 44 is configured to transmit at least part of the captured video saved in the memory 40 along with the corresponding position.
In some cases, the mobile security device 32 is configured to be carried or even fly indoors, and the position sensor 38 is configured to enable the controller 44 to ascertain its indoors position via triangulation. For example, the position sensor 38 may be configured to be able to triangulate between multiple beacons that are disposed within an indoors facility. The position sensor 38 may be configured to use a 5G cellular network to ascertain its position. In some cases, the position sensor 18 may be configured to triangulate its position using a magnetometer, ultrawide band (UWB) or even BLE (Bluetooth low energy). The position sensor 38 may have access to a floorplan of the facility by communicating with a BIM (building information model) or even a BMS (building management system). For a mobile security device 32 that is configured to fly outdoors, GPS (global positioning system) may be used to ascertain position.
In this, position may refer strictly to the physical location of the mobile security device 32. For example, the position of the mobile security device 32 is 14 meters due north of door #6 within the facility. In some cases, position may also take into account the field of view of the video camera 36. While the mobile security device 32 may currently be 14 meters due north of door #6 within the facility, the video camera 36 may have a field of view that extends into the lobby. In some cases, position can be a combination of physical location and field of view.
In the example shown, the remote device 34 includes a memory 46 that is configured to store a plurality of video analytics algorithms 48. The illustrative remote device 34 includes a transceiver 50 and a controller 52 that is operably coupled with the memory 46 and the transceiver 50. The video analytics algorithms 48 include a number of different video analytics algorithms. The video analytics algorithms may including but are not limited to a people count video analytics algorithm, a crowd detection video analytics algorithm, a loitering detection video analytics algorithm, an intrusion detection video analytics algorithm, a queue length video analytics algorithm, an unidentified object detection video analytics algorithm, an occupancy detection video analytics algorithm, a vehicle detection video analytics algorithm, a vehicle count video analytics algorithm and a license plate detection video analytics algorithm. These are just examples.
The controller 52 of the remote device 34 is configured to receive the at least part of the captured video and the corresponding position transmitted by the mobile security device 32. The controller 52 of the remote device 34 is configured to select one or more video analytics algorithms of the plurality of video analytics algorithms 48 based at least in part upon the received position of the mobile security device 32 and to perform the selected one or more video analytics algorithms on the received captured video, resulting in one or more video analytics results. The controller 52 of the remote device 34 is configured to store the one or more video analytics results.
In some cases, selecting one or more video analytics algorithms includes selecting a single video analytics algorithm out of the plurality of video analytics algorithms 48. In some cases, the remote device 34 is a remote server, and the plurality of video analytics algorithms 48 include one or more of a face detection video analytics algorithm, a facial recognition video analytics algorithm, a mask detection video analytics algorithm, and a walking gate video analytics algorithm.
In some instances, the controller 44 of the mobile security device 32 is configured to select one or more mobile security device video analytics algorithms of a plurality of mobile security device video analytics algorithms based at least in part upon the position of the mobile security device 32. The controller 44 of the remote device 34 may be configured to perform the selected one or more mobile security device video analytics algorithms on the captured video, resulting in one or more video analytics results, and to transmit one or more of the video analytics results to the remote device 34.
In some cases, the video is captured by a mobile security device, and at least some of the selected one or more video analytics algorithms are performed by the mobile security device. In some instances, the video is captured by a mobile security device, and the mobile security device is configured to transmit at least part of the captured video and the location indicator to a remote device, and at least some of the selected one or more video analytics algorithms are performed by the remote device.
Those skilled in the art will recognize that the present disclosure may be manifested in a variety of forms other than the specific embodiments described and contemplated herein. Accordingly, departure in form and detail may be made without departing from the scope and spirit of the present disclosure as described in the appended claims.
