Patent Application
20180096576
The present subject-matter relates to a sensor system, camera, and a system integrating the camera and sensor system configured to detect a person.
A camera may be used to acquire information about a place or an object. The information is visual image data generated by the camera corresponding to the scene falling with the field of view of the camera. A sensor system, such as a radar or depth device, may be used to acquire other information about a place or an object.
Some currently available security systems include means of detecting loitering, often based on tracking the movement of an object within a fixed area for a period of time. However typical video analytics systems fail to detect an immobile person, who simply becomes part of the background image. Security systems available are unable to detect sleeping persons as they do not move sufficiently to register on a camera system; instead they are treated like immobile objects. This poses problems in situations in which the object being monitored is at rest and moving only minimally. An example of such a situation occurs in certain areas, such as ATM vestibules, which are attractive and relatively warm areas for sleeping compared to the outdoors.
Currently banks and other operators of publicly accessible areas, for example ATM vestibules, have difficulty detecting and moving sleeping persons from their property. The presence of persons loitering or sleeping in a vestibule may discourage customers from entering the vestibule.
Another challenge for video cameras is detecting objects or persons in multiple rooms, or in situations where privacy needs to be protected, for example changing rooms or showers.
The embodiments described herein provide in one aspect a combined video surveillance system including a sensor system and a camera device. The camera device may include at least one camera module operable for capturing a scene corresponding to a field of view of the camera and for generating image data of the captured scene, a transceiver operable for receiving the access data transmitted from the battery-powered physical access control device, a networking module operable for transmitting the image data and access data to an external networked device, a wireless power transmitter operable for transmitting power wirelessly to the wireless power receiver of the access control device, and at least one power supply operable for receiving power over a wired connection and supplying power to the camera, the transceiver, the network module and the wireless power transmitter.
The embodiments described herein provide another aspect of a camera device. The camera device includes at least one camera module operable for capturing a scene corresponding to a field of view of the camera module and for generating image data of the captured scene, at least one wireless power transmitter operable for transmitting wireless power to a battery-powered external sensing device, a transceiver operable for receiving from the external sensing device sensed data pertaining to at least one condition sensed by the external sensing device, a networking module operable for transmitting the image data and data pertaining to the at least one sensed condition to an external networked device, and at least one power supply operable for receiving power over a wired connection and supplying power to the camera module, the transceiver, the networking module and the wireless power transmitter.
According to some example embodiments, the networking module transmits the image data and the access data to the external networked device over the wired connection.
According to some example embodiments, the camera device further includes a video analytics module operable for performing video analytics on the image data and determining an occurrence of a video analytics event based on a combination of one or more results of the performed video analytics and access data received from the access control device.
According to some example embodiments, the transceiver of the camera device is further operable for receiving information pertaining to a battery status of the at least one battery of the access control device and the wireless power transmitter is operable for adjusting the transmission of wireless power to the access control device based on the received information pertaining to the battery status.
According to some example embodiments, an effective powered space of the wireless power transmitter of the camera device substantially overlaps with the field of view of the camera module.
According to some example embodiments, the camera module is pivotable and wherein the wireless power transmitter pivots with the camera module to maintain overlap of the power coverage cone with the field of view.
According to some example embodiments, the camera device comprises a plurality of wireless power transmitters, each transmitter being operable for directionally transmitting wireless power.
According to some example embodiments, at least one of the plurality of wireless power transmitters is pivotable to change the space occupied by its power coverage cone.
According to some example embodiments, the sensor system includes a radar system configured to determine the presence of a person.
According to some example embodiments, the sensor system includes a radar system or a depth camera, and the sensor system, on determination of the presence of a person, provides an alert.
According to some example embodiments, a method to detect a person loitering in an area under surveillance is provided, including: generating a model of the background of the area under surveillance for a depth sensor; obtaining data maps of the area under surveillance over a period of time from the depth sensor; comparing the data maps with the model of the background to generate foreground frames over the period of time; and detecting a person loitering in the area under surveillance from analyzing the foreground frames over the period of time.
According to some example embodiments, a security system for detecting a person loitering in an area under surveillance is provided, including a depth sensor configured to monitor the area under surveillance and a processor coupled to the depth sensor and configured to: generate a model of the background of the area under surveillance for the depth sensor; obtain data maps of the area under surveillance over a period of time from the depth sensor; compare the data maps with the model of the background to generate foreground frames over the period of time; and detect a person loitering in the area under surveillance from analyzing the foreground frames over the period of time.
According to some example embodiments, a security system for detecting a person loitering in an area under surveillance is provided, including a depth sensor configured to monitor the area under surveillance; a radar transceiver configured to monitor the area under surveillance, the radar system configured to transmit radar signals to and receive radar signals from the area under surveillance; a processor coupled to the depth sensor and configured to: generate a model of the background of the area under surveillance for the depth sensor; obtain data maps of the area under surveillance over a period of time from the depth sensor; compare the data maps with the model of the background to generate foreground frames over the period of time; and detect a person loitering in the area under surveillance from analyzing the foreground frames over the period of time; and the processor coupled to the radar transceiver and configured to: determine if the radar signals indicate a presence of a person in the area under surveillance; on determination of the presence of the immobile person, determine if the immobile person is present in the area under surveillance for a predetermined time period; and on determination of the presence of the person for a time period greater than the predetermined time period, providing instructions to cause an alert.
According to some example embodiments, a computer-implemented method for detecting a person in a publicly accessible location is provided, including: providing a radar system positioned to monitor the publicly accessible location, the radar system configured to transmit radar signals to and receive radar signals from the location; determining if the radar signals indicate a presence of a person in the location; and on determination of the presence of the person, causing an alert.
According to some example embodiments, a method for controlling loitering in an unsecured location is provided, including: providing a radar transceiver in a secured location, the radar transceiver configured to transmit radar signals to and receive radar signals from the unsecured location; determining if the received radar signals indicate a presence of a first person in the unsecured location; if the first person is present, determining if the first person is present in the unsecured location for a predetermined time period; and if the first person is present in the unsecured time period for a time greater than the predetermined time period, providing an alert, report or alarm.
According to some example embodiments, a physical security system is provided, including: a radar transceiver configured to monitor a publicly accessible location, the radar system configured to transmit radar signals to and receive radar signals from the publicly accessible location; a processor coupled to the radar transceiver and configured to: determine if the radar signals indicate a presence of a person in the publicly accessible location; on determination of the presence of the immobile person, determine if the immobile person is present in the unsecured location for a predetermined time period; on determination of the presence of the person for a time period greater than the predetermined time period, providing instructions to cause an alert.
According to some example embodiments, a physical security system for monitoring a plurality of rooms is provided, including: a radar transceiver configured to monitor the plurality of rooms, the radar transceiver configured to transmit radar signals to and receive radar signals from each room in the plurality of rooms; a processor communicatively coupled to the transceiver and configured to determine if each of said rooms is occupied.
The processor may be further configured to determine if a detected occupant is asleep or in distress, or to determine the length of time in which the occupant has been present in a room. The rooms may be cells or changing rooms, or may contain one or more animals.
According to some sample embodiments, a method for controlling loitering in a room within a plurality of rooms is provided, including: providing a radar transceiver, the radar transceiver configured to transmit radar signals to and receive radar signals from each room within the plurality of rooms; determining if the received radar signals indicate a presence of a first person in the room; if the first person is present, determining if the first person is present in the room for a predetermined time period; and if the first person is present in the room for a time greater than the predetermined time period, providing an alert, report or alarm.
According to some sample embodiments, a computer-implemented method for detecting a person in a room amongst a plurality of rooms is provided, including: providing a radar system positioned to monitor each room amongst the plurality of rooms, the radar system configured to transmit radar signals to and receive radar signals from each room; determining if the radar signals indicate a presence of a person in one or more of the rooms amongst the plurality of rooms; and on determination of the presence of the person, causing the generation of a report comprising identification of the rooms in which the presence of a person was indicated.
The detailed description refers to the following figures, in which:
It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity.
Furthermore, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements.
Directional terms such as “top”, “bottom”, “upwards”, “downwards”, “vertically”, and “laterally” are used in the following description for the purpose of providing relative reference only, and are not intended to suggest any limitations on how any article is to be positioned during use, or to be mounted in an assembly or relative to an environment. Additionally, the term “couple” and variants of it such as “coupled”, “couples”, and “coupling” as used in this description is intended to include indirect and direct connections unless otherwise indicated. For example, if a first device is coupled to a second device, that coupling may be through a direct connection or through an indirect connection via other devices and connections. Similarly, if the first device is communicatively coupled to the second device, communication may be through a direct connection or through an indirect connection via other devices and connections.
The terms “an aspect”, “an embodiment”, “embodiment”, “embodiments”, “the embodiment”, “the embodiments”, “one or more embodiments”, “some embodiments”, “certain embodiments”, “one embodiment”, “another embodiment” and the like mean “one or more (but not all) embodiments of the disclosed invention(s)”, unless expressly specified otherwise. A reference to “another embodiment” or “another aspect” in describing an embodiment does not imply that the referenced embodiment is mutually exclusive with another embodiment (e.g., an embodiment described before the referenced embodiment), unless expressly specified otherwise.
The terms “including”, “comprising” and variations thereof mean “including but not limited to”, unless expressly specified otherwise.
The terms “a”, “an” and “the” mean “one or more”, unless expressly specified otherwise.
The term “plurality” means “two or more”, unless expressly specified otherwise. The term “herein” means “in the present application, including anything which may be incorporated by reference”, unless expressly specified otherwise.
The term “e.g.” and like terms mean “for example”, and thus does not limit the term or phrase it explains.
The term “respective” and like terms mean “taken individually”. Thus if two or more things have “respective” characteristics, then each such thing has its own characteristic, and these characteristics can be different from each other but need not be. For example, the phrase “each of two machines has a respective function” means that the first such machine has a function and the second such machine has a function as well. The function of the first machine may or may not be the same as the function of the second machine.
Where two or more terms or phrases are synonymous (e.g., because of an explicit statement that the terms or phrases are synonymous), instances of one such term/phrase does not mean instances of another such term/phrase must have a different meaning. For example, where a statement renders the meaning of “including” to be synonymous with “including but not limited to”, the mere usage of the phrase “including but not limited to” does not mean that the term “including” means something other than “including but not limited to”.
Neither the Title (set forth at the beginning of the first page of the present application) nor the Abstract (set forth at the end of the present application) is to be taken as limiting in any way as the scope of the disclosed invention(s). An Abstract has been included in this application merely because an Abstract of not more than 150 words is required under 37 C.F.R. Section 1.72(b) or similar law in other jurisdictions. The title of the present application and headings of sections provided in the present application are for convenience only, and are not to be taken as limiting the disclosure in any way.
Numerous embodiments are described in the present application, and are presented for illustrative purposes only. The described embodiments are not, and are not intended to be, limiting in any sense. The presently disclosed aspect(s) are widely applicable to numerous embodiments, as is readily apparent from the disclosure. One of ordinary skill in the art will recognize that the disclosed aspect(s) may be practiced with various modifications and alterations, such as structural and logical modifications. Although particular features of the disclosed aspect(s) may be described with reference to one or more particular embodiments and/or drawings, it should be understood that such features are not limited to usage in the one or more particular embodiments or drawings with reference to which they are described, unless expressly specified otherwise.
No embodiment of method steps or product elements described in the present application is essential or is coextensive, except where it is either expressly stated to be so in this specification or expressly recited in a claim.
“Battery” herein refers to not only a device in which chemical energy is converted into electricity and used as a source of power, it also refers to any alternatively suitable energy storage devices such as, for example, a capacitor of suitable size and construction.
“Image data” herein refers to data produced by a camera device and that represents images captured by the camera device. The image data may include a plurality of sequential image frames, which together form a video captured by the camera device. Each image frame may be represented by a matrix of pixels, each pixel having a pixel image value. For example, the pixel image value may be a numerical value on grayscale (e.g. 0 to 255) or a plurality of numerical values for colored images. Examples of color spaces used to represent pixel image values in image data include RGB, YUV, CYKM, YCbCr 4:2:2, YCbCr 4:2:0 images. It will be understood that “image data” as used herein can refer to “raw” image data produced by the camera device and/or to image data that has undergone some form of processing. It will be further understood that “image data” may refer to image data representing captured visible light in some examples and may refer to image data representing captured depth information and/or thermal information in other examples.
“Processing image data” or variants thereof herein refers to one or more computer-implemented functions performed on image data. For example, processing image data may include, but is not limited to, image processing operations, analyzing, managing, compressing, encoding, storing, transmitting and/or playing back the video data. Analyzing the image data may include segmenting areas of image frames and detecting objects, tracking and/or classifying objects located within the captured scene represented by the image data. The processing of the image data may cause modified image data to be produced, such as compressed and/or re-encoded image data. The processing of the image data may also cause additional information regarding the image data or objects captured within the images to be outputted. For example, such additional information is commonly understood as metadata. The metadata may also be used for further processing of the image data, such as drawing bounding boxes around detected objects in the image frames.
Referring now to
The camera device 10 includes one or more processors, one or more memory devices coupled to the processors and one or more network interfaces. The memory device can include a local memory (e.g. a random access memory and a cache memory) employed during execution of program instructions. The processor executes computer program instruction (e.g., an operating system and/or application programs), which can be stored in the memory device.
In various embodiments the processor may be implemented by any processing circuit having one or more circuit units, including a digital signal processor (DSP), graphics processing unit (GPU) embedded processor, etc., and any combination thereof operating independently or in parallel, including possibly operating redundantly. Such processing circuit may be implemented by one or more integrated circuits (IC), including being implemented by a monolithic integrated circuit (MIC), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), etc. or any combination thereof. Additionally or alternatively, such processing circuit may be implemented as a programmable logic controller (PLC), for example. The processor may include circuitry for storing memory, such as digital data, and may, for example, include the memory circuit or be in wired communication with the memory circuit.
In various example embodiments, the memory device is communicatively coupled to the processor circuit and is operable to store data and computer program instructions. Typically, the memory device is all or part of a digital electronic integrated circuit or formed from a plurality of digital electronic integrated circuits. The memory device may be implemented as Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash memory, one or more flash drives, universal serial bus (USB) connected memory units, magnetic storage, optical storage, magneto-optical storage, etc. or any combination thereof, for example. The memory device may be operable to store memory as volatile memory, non-volatile memory, dynamic memory, etc, or any combination thereof.
In various example embodiments, a plurality of the components of the image capture device may be implemented together within a system on a chip (SOC). For example, the processor, the memory device and the network interface may be implemented within a SOC. Furthermore, when implemented in this way, both a general purpose processor and DSP may be implemented together within the SOC.
The camera device 10 includes at least one camera module 16 (for convenience of illustration only one is shown in the illustrated example embodiment) that is operable to capture a plurality of images and produce image data representing the plurality of captured images. The camera module 16 generally refers to the combination of hardware and software sub-modules that operate together to capture the plurality of images of a scene. Such sub-modules may include an optical unit (e.g. camera lens) and an image sensor. In the case of a digital camera module, the image sensor may be a CMOS, NMOS, or CCD type image sensor.
The lens and sensor combination defines a field of view. When positioned at a given location and at a given orientation, the camera module 16 is operable to capture the real-life scene falling within the field of view of the camera and to generate image data of the captured scene.
The camera module 16 may perform some processing of captured raw image data, such as compressing or encoding the raw image data.
The camera device 10 may optionally include a video analytics module 24. The video analytics module 24 receives image data from the camera module 16 and analyzes the image data to determine properties or characteristics of the captured image or video and/or of objects found in scene represented by the image or video. Based on the determinations made, the video analytics module 24 may further output metadata providing information about the determinations. Examples of determinations made by the video analytics module 24 may include one or more of foreground/background segmentation, object detection, object tracking, object classification, virtual tripwire, anomaly detection, facial detection, facial recognition, license plate recognition, identifying objects “left behind”, monitoring objects (i.e. to protect from stealing), and business intelligence. However, it will be understood that other video analytics functions known in the art may also be implemented by the video analytics module 24.
The camera device 10 may optionally include a video management module 32. The video management module 32 receives image data and performs processing functions on the image data related to video transmission, playback and/or storage. For example, the video management module 32 can process the image data to permit transmission of the image data according to bandwidth requirements and/or capacity. The video management module 32 may also process the image data according to playback capabilities of a client device that will be playing back the video, such as processing power and/or resolution of the display of the client device. The video management 32 may also process the image data according to storage capacity in the camera device 10 or in other devices connected to the camera device 10 over a network.
The camera device 10 may optionally include a set 40 of storage modules. For example, and as illustrated, the set 40 of storage modules include a video storage module 48 and a metadata storage module 56. The video storage module 48 stores image data, which may be image data processed by the video management module 32. The metadata storage module 56 stores information data output from the video analytics module 24.
It will be understood that while video storage module 48 and metadata storage module 56 are illustrated as separate modules, they may be implemented within a same hardware storage device whereby logical rules are implemented to separate stored video from stored metadata. In other example embodiments, the video storage module 48 and/or the metadata storage module 56 may be implemented within a plurality of hardware storage devices in which a distributed storage scheme may be implemented.
The storage modules 48, 56 provide non-transitory storage of image data and/or metadata. In other example embodiments wherein storage modules 48, 56 are not provided, image data generated by the camera module 16 and metadata generated by the video analytics module 24 may be immediately transmitted to an external device over a network.
The camera device 10 includes a networking module 64 operable for providing data communication with another device over a network 72. The network 72 may be a local area network, an external network (e.g. WAN, Internet) or a combination thereof. In other examples, the network 72 may include a cloud network.
The camera device 10 further includes a transceiver 80 operable for communicating wirelessly with another device. The wireless communication may be provided according to any protocol known in the art, such as Bluetooth, Wi-Fi, ZigBee or cellular communication. Alternatively, camera device 10 may be operable for communicating over a wire with another device.
In some examples, the transceiver 80 is a short-range, low-power transceiver. A short-range, low-power transceiver may be useful for reducing power consumption of the external device with which the camera device 10 is communicating. For example, the transceiver 80 that is short-range may have a communication range of less than about 10 meters. For example, the transceiver 80 that is low-power may have a power consumption of less than about 0.5 Watts. A short-range, low-power transceiver may implement a low-energy Bluetooth or low-energy Wi-Fi protocol known in the art.
The camera device 10 further includes a wireless power transmitter 88 that is operable for transmitting power wirelessly to an external electrical load. The external electrical load may be an energy storage device, such as at least one battery or capacitor. For example, power may be transferred by magnetic fields in which one or more coils of wires in the wireless power transmitter 88 is coupled by magnetic induction with a cooperating coil in the external device that is being powered by the wireless power. The inductive coupling between the wireless power transmitter 88 and a cooperating device receiving the power may be resonant inductive coupling or electrodynamic induction. It will be understood that the wireless power transmission is not limited to non-radiative techniques. In some examples, longer range techniques may be used, such as power beaming based on microwaves or lasers.
The camera device 10 further includes a power supply 96 operable for supplying electrical power to the hardware components of the camera device 10, such as those implementing the camera module 16, transceiver 80, networking module 64 and wireless power transmitter 88.
In some examples, the power supply 96 receives electrical power from a power source over a wired connection. The power source may be mains electricity (ex: 110V/220V AC), which may be converted to a supply suitable for the camera device 10 (ex: converting to DC, rectifying to a lower voltage). In some alternative examples, the power source may be an intermediate device that supplies power in addition to performing another function, such as processing or networking. In yet further alternative examples, the power supply may be supplying power in a sustainable manner based on, for instance, solar power technology or power received wirelessly from another device in communication with the camera device 10.
In one example embodiment, power may be supplied to the power supply 96 over a connection that is also providing data communication. For example, power may be supplied to the power supply 96 by power over Ethernet (POE), wherein the cable connected to the networking module 64 for network data communication is also used for supplying power to the power supply. As illustrated, the same cable 104 that is connected to the network (e.g. connected to a network switch or router) is also connected to the power supply 96.
The camera device 10 may further include a power management module 112 that is operable for managing the supply of power from the power supply 96 to various hardware components of the camera device 10. The power management module 112 may manage the power being consumed by the wireless power transmitter 88 separately from management of power being consumed by other components of the camera device 10. The power management module 112 may further control the priority of providing power to various modules of the camera device 10. This prioritization in the case of high power demand from various modules, which may otherwise cause system overload.
For example, a wireless power transmitter power management submodule may control the power level of the wireless power transmitted from the wireless power transmitter 88. The power level may be varied according to characteristics of an external device receiving the wireless power. Such characteristics may include one or more of the distance of the external device from the camera device 10, the average power requirement of the external device, the instantaneous power requirement of the external device, and the current battery status of the external device.
The power level may also be varied according to environmental factors, such as time of day, location, and number of proximately located devices. For example, where the camera device 10 is used for charging the external device, the wireless power transmitter power management submodule may choose to transmit wireless power for charging during off-peak hours.
The power level may also be varied according to power load requirements from other components of the camera device 10. For example, during periods when other components of the camera device 10 experience heavy load, the power management module 112 may supply less or no power to the wireless power transmitter. These periods may occur when the camera device 10 has to handle a large amounts of data, such as transferring or backing up data stored within the storage module 40.
The example camera device 10 is suitable for use in conjunction with an external device that requires data communication with another device over a network and that would benefit from receiving wirelessly transmitted power. The camera device 10 can provide network connectivity to the external device via data communication provided between the wireless transceiver 80 of the camera device 10 and a corresponding wireless transceiver of the external device. The network connectivity is further provided through the connection of the networking module 64 of the camera device 10 with the network 72. Accordingly, the external device may be in communication another network node connected to the network 72 only via the camera device 10 and without requiring some form of wired and/or wireless connection from the external device to the network 72.
The camera device 10 can further provide a continued power source for the external device via wireless power transmitter 88 transmitting power to the external device. The external device may be battery-operated and the power transmitted wirelessly from the camera device 10 may be used to charge at least one battery of the external device. Accordingly, the external device may operate without having to receive power over a wired power cable. Furthermore, even where the external device may be fully battery-operated, the providing of wireless power from the camera device 10 to the external device to charge the battery of the external device may eliminate, or reduce the frequency, of having to change the battery.
In some example embodiments, the power output from the wireless power transmitter 88 may be variably controlled. For example, the level of power output may be adjusted according to the power consumption of the external device receiving the wirelessly transmitted power. The level of power out may also be adjusted based on one or more parameters of the deployment of the camera device 10 with the external device, such as the distance there between. The power output from the wireless power transmitter 88 may be adjusted so that the level of wireless power received at the external device corresponds with a power requirement of the external device, such as an average power requirement of the external device. The power output may also be adjusted based on a change in power requirement of the external device. However, the power output from the wireless power transmitter 88 may be throttled by the power management module 112 to ensure continued proper functioning of the camera device 10. In some example embodiments, the wireless power transmitter 88 may implement trickle charging or slow charging of the external device.
In some example embodiments, the wireless power transmitter 88 may be chosen to provide at least 3 watts power to a power-receiving external device located at a distance of at most 10 meters from the camera device 10. For example, such a power output would effectively charge a depth sensor, radar sensor, or typical PIR motion sensor.
In other example embodiments, the wireless power transmitter 88 may be chosen to provide substantially less power, such as about 0.2 mW of power at a distance of at most 10 meters from the camera device 10. This level of power output is suitable for external devices that are typically on standby, such as a smoke alarm.
Referring now to
The sensor system 208 may be a radar sensor, a depth sensor, or both, as described below. Sensor system 208 includes a sensor 216.
The sensor system 208 may include a storage module 224. The storage module 224 may be operatively connected with sensor 216 to receive sensed signals and store the sensed signals. The storage module 224 may also store one or more sensing rules. The sensor 216 may implement sensing based on applicable sensing rules. For example, the rules may cause the sensor 216 to cease sensing during given periods of the day, for example during daylight, and carry out sensing at other periods of the day, for example during the night, when persons are most likely to be loitering or sleeping in an ATM vestibule.
The sensor system 208 includes a communications transceiver 232 operable for providing data communication with the camera device 10 via the transceiver 80. The communications transceiver 232 of the sensor system 208 may implement a wireless communication protocol that is compatible with the communication protocol implemented by the transceiver 80 of the camera device 10. For example, the communications transceiver 232 may also be a short-range, low-power transceiver.
Sensed signals generated by the sensor 216 can be transmitted from sensor system 208 using its communications transceiver 232 and received at the camera device 10 using its transceiver 80. The sensed data may be further transmitted to external network device 264 from the camera device 10 over the network 72.
The sensor system 208 may further receive commands from the camera device 10. The commands may have been initially transmitted from the external network device 264 to the camera device 10 via the network 72 and the networking module 210 of the camera 10. For example, the commands may be for controlling the sensor system 208, such as commands for changing sensing rules applied to the sensor system 208.
The sensor system 208 further includes a wireless power receiver 240 that is operable for receiving power transmitted wirelessly from the wireless power transmitter 88 of the camera device 10. The wireless power receiver 240 is configured to be compatible with the wireless power transmitter 88 of the camera device 10. For example, the wireless power receiver 240 includes one or more coil of wires in which a flow of electrical current is induced by the wireless power transmitted from the camera device 10.
The sensor system 208 may further include at least one battery 248 or other suitable form of power storage device for supplying power to one or more components of the sensor system 208. The at least one battery 248 may supply power to the sensor 216, and the communications transceiver 232. The at least one battery 248 is rechargeable using power transmitted wirelessly from the camera device 10 and received by the wireless power receiver 240.
The sensor system 208 may further include a battery management module 256. The battery management module 256 operates to manage charging of the at least one battery 248 using the power received by the wireless power receiver 240.
In one example embodiment, the battery management module 256 may sense the charge level of the at least one battery 248 and implements charging of the battery 248 when the charge level falls below a predetermined level.
In another example embodiment, the battery management module 256 may implement charging of the battery 248 any time wireless power is available from the wireless power receiver 240. The battery management module 256 may be further operable to implement trickle charging or slow charging of the battery 248.
In yet another example embodiment, the battery management module 256 may be further operable to sense the battery charge level and to communicate the charge level to the camera device 10 using the wireless transceiver 232. The camera device 10 may be configured to transmit wireless power only when it receives an indication that the charge level of the at least one battery 248 of the radar device 208 has fallen below a predetermined level. Additionally, or alternatively, the battery management module 256 may transmit, using the wireless transceiver 232, a request to the camera device 10 to begin wireless transmission of power to the sensing device so that the power can be used for charging the at least one battery.
Continuing with
In various example embodiments, the camera device 10 may be configured to transmit the sensed signals received from the sensor system 208 to the same destination networked device 264 over the network 72. For example, the destination networked device 264 may be a server that processes or manages the image data and/or the sensed signals. When being transmitted to the same destination networked device, image data that is captured by the camera module 16 at a given time is logically associated with sensed signals pertaining to one or more conditions sensed by the sensor 216 at the same time. “Logically associated” herein refers to an association in which knowledge of the relevant image data allows retrieval of its logically associated sensed signals and vice versa. For example, the image data and its corresponding signal may both include a time stamp, which provides the logical association.
According to various example embodiments wherein the camera device 10 is used in a video surveillance application to visually monitor an area or asset, such as an ATM vestibule, the condition sensed by the sensor system 208 may provide information about the area or asset, which may provide enhanced monitoring. For example, the radar signals or depth camera images may be used to confirm or provide further information regarding an event that is captured by the camera device 10. This information may be also be used to confirm or improve certainty of a determination made by the video analytics module 24.
In some example embodiments, the video analytics module 24 may determine properties or characteristics of the captured image or video and/or of objects found in the scene represented by the image or video based on a combination of analysis of the image data and one or more relevant signals from sensor system 208. Relevant signals sensed by the sensor system 208 may be conditions sensed during a time period corresponding to the time period of the image data being analyzed.
According to various example applications, the sensor system 208 is located in proximity of the camera device 10, such as within the same physical area. For example, the sensor system 208 may be located such that signals received by the sensor system 208 are relevant to the image data captured by the camera device 10. Accordingly, the signals received may serve to enhance the monitoring performed using the camera device 10. It will be appreciated that the proximity of the camera device 10 with the sensor system 208 allows for effective wireless transmission of power from camera device 10 to the sensor system 208 and for effective wireless data communication between the camera device 10 and the sensor system 208. This allows the sensor system 208 to operate fully wirelessly (i.e. without requiring a wired connection for data communication with an external device and for receiving power). It will be further appreciated that even in other examples where the sensor system 208 generates signals that is not pertinent to the image data captured by the camera device 10, the interaction between the camera device 10 and the sensing device 10 allows the sensor system 208 to operate fully wirelessly.
Referring now to
The plurality of power transmitters may each be used to power different sets of one or more sensor systems 208 and/or access control devices 308. For example, the sensor systems 208 may be sparsely located such that a single power transmitter 88 cannot effectively provide power to all of the sensor systems 208. The wireless transceiver 80 of the alternative camera device 10′ may be used for simultaneous data communication with a plurality of sensor systems 208 to which the alternative camera device 10′ is transmitting power. For example, an appropriate multiplexing scheme may be used to maintain data communication with each of the plurality of sensor systems 208.
In one example embodiment, at least one of the wireless power transmitters 88 of the alternative camera device 10′ is pivotable so as to change an orientation of the wireless power transmitters 88. For example, a wireless power transmitter 88 may be angularly pivoted to be sufficiently aligned with an external sensor system 208 so as to effectively transmit power to that external device.
Referring now to
Referring now to
The system 400, as depicted, includes at least one camera device. In the example illustrated in
In the illustrated example, the third camera device 10c is connected to the network 72 through a processing appliance 404. The processing appliance 404 is operable to process the image data output by the third camera device 10c. The processing appliance 404 includes one or more processors and one or more memory devices coupled to the processor. The processing appliance 404 may also include one or more network interfaces.
The first camera device 10a is in data communication with a first sensor system 208a using their respective wireless transceivers 80, 232. Signals and data generated by the sensor system 208a is transmitted over the network 72 via the first camera device 10a. The first camera device 10a further transmits power wirelessly from its wireless power transmitter 88. The transmitted power is received at the sensing device 240 of the first sensor system 208a, which may be used to charge its one or more batteries or energy storage devices.
The second camera device 10b is in data communication with a second sensor system 208b. It will be understood that although two antennas are illustrated, a single wireless transceiver 80 in the second camera device 10b may be in data communication with the second sensor system 208b. Sensed signals and data generated by the second sensor system 208b is transmitted over the network 72 via the second camera device 10b.
In another example embodiment, the second camera device 10b is multi-transmitter device, as described herein with reference to
The third camera device 10c is not transmitting wireless power. The third camera device 10c may be a standard IP camera that does not have wireless power transmission capabilities.
The system 400 may further include a third sensing device 208c, which may be an environmental sensing device, which is in direct connection with the network 72.
The system 400 includes at least one workstation 408 (e.g. server), each having one or more processors. The at least one workstation 408 may also include storage memory. The workstation 408 receives image data from at least one camera device 10 and performs processing of the image data. The workstation 408 may further send commands for managing and/or controlling one or more of the camera devices 10. The workstation 408 may receive raw image data from the camera device 10. Alternatively, or additionally, the workstation 408 may receive image data that has already undergone some intermediate processing, such as processing at the camera device 10 and/or at a processing appliance 404. The workstation 408 may also receive metadata from the image data and perform further processing of the image data.
The video capture and playback system 400 further includes at least one client device 164 connected to the network 72. The client device 164 is used by one or more users to interact with the system 400. Accordingly, the client device 164 includes at least one display device and at least one user input device (for example, mouse, keyboard, touchscreen, joy stick, microphone, gesture recognition device, etc.). The client device 164 is operable to display on its display device a user interface for displaying information, receiving user input, and playing back images and/or video. For example, the client device may be any one of a personal computer, laptops, tablet, personal data assistant (PDA), cell phone, smart phone, gaming device, and other mobile and/or wearable devices.
Referring back to
According to various example embodiments, the field of view of the camera device 10 substantially overlaps with the effectively powered space of the wireless power transmitter 88. The field of view of the camera device 10 may be fully encompassed within the effectively powered space of wireless power transmitter 88. The field of view of the camera device 10 may be fully encompassed in that the effectively powered space occupies a larger space than the field of view. However, it will be understood that the field of view of the camera device 10 may extend past the outer limit of the effectively powered space based on a distance from the camera device 10.
By ensuring that the field of view of the camera device 10 is fully encompassed within the effectively powered space of the wireless power transmitter 88, any object that falls within the field of view will also be within effectively powered space of the wireless power transmitter 88 and can receive wireless power therefrom (so long as the distance of the object does not exceed the outer limit of the operational space). This may facilitate installation of a sensor system 208 in that the installer only needs to place the sensor system 208 within the field of view of the camera device 10 to ensure that the sensor system 208 will be properly receiving wireless power from the camera device.
According to various example embodiments wherein the optical unit of the camera device 10 is pivotable to change the field of view of the camera device 10, the wireless power transmitter 88 is configured to maintain the overlap of the field of view with the operational space. The wireless power transmitter 88 may be configured to maintain the field of view being fully encompassed with the effectively powered space over the range of pivotable motion of the optical unit of the camera. Examples of cameras with a pivotable optical unit include a dome camera and a pan-tilt-zoom (PTZ) camera.
In one example embodiment, the wireless power transmitter 88 transmits power directionally. Accordingly, the operational space of the wireless power transmitter is defined by an effective power coverage cone. The wireless power transmitter 88 and the optical unit of the camera module 16 may be substantially aligned so that the field of view of the camera device 10 overlaps with the power coverage cone. The alignment may be such that the field of view of the camera device is fully encompassed within the power coverage cone.
Referring now to
Referring now to
Referring now to
Radar system hub 350 may be mounted above the ceiling of the location being monitored, and includes a processor 360 to configure and collect data received from radar systems 302A, 302B, 302C, and 302D. If a presence is detected by one of radar systems 302A, 302B, 302C, and 302D a relay system 370 in radar system hub 350 is energized and relays a message to camera device 10 and/or the network 72.
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Radar system 300 works by transceiver 906 sending and receiving radar signals. The returning signal will indicate the distance to a detected objected and the Doppler Effect is used to determine a portion of the velocity of the detected object as indicated by the change in frequency of the returned radar signal as determined using a Fourier transformation. Comparing signals over time allows processor 902 to determine the direction of the detected object's motion.
Radar system 300 may be used for a number of purposes, including identifying the presence of a person in a location, such as a dressing room, a prison cell, or ATM vestibule, by detecting biometric indicators such as breathing or heartbeats. Detection of a human being as a living object, and not as a motionless object, can be performed by short-range radars using microwave signals ranging in frequency, waveform, duration, and bandwidth. Radar system 300 can detect people not actively moving, only breathing and with a heartbeat, and thereby determine the presence of a sleeping person.
On reflection from a person, a radar signal acquires specific biometrical modulation, which does not occur in reflections from inanimate objects. This modulation is produced by heartbeats, pulsations of vessels, lungs, and skin vibrations in the region of the person's thorax and larynx, which occur synchronously with breathing motions and heartbeat. These processes are nearly periodic, with typical frequencies in the range of 0.8−2.5 Hz for heartbeat and 0.2−0.5 Hz for breathing. Therefore, the delay or phase of the reflected signal is periodically modulated by these periodic oscillations. The modulation parameters are thus determined by the frequencies and intensities of respiration and heartbeat.
The sensitivity of radar probing in the gigahertz band may reach 10−9 m. In practice, radar probing of live persons is performed against the background of reflections from local objects; as a rule, the intensity of these reflections exceeds the intensity of signals from a human object. Human objects, however, are distinguished by periodic and aperiodic modulation synchronous with the respiration and heartbeat of a person. Modulation of this type is either absent in signals reflected from local objects or has different time and spectral characteristics. This allows for recognition of signals reflected by a human person against the background reflections from local objects.
Radar systems 300 may use probing signals of different types, for example unmodulated monochromatic signals, UWB video pulses, and wideband SFM signals. The main advantage of wideband and UWB signals over monochromatic signals is that they allow the range separation of targets from exterior interference, such as reflections from local objects.
Referring now to
The biometric signals received can also be used to detect if the person is asleep or not, or is undergoing a health emergency (for example has an erratic heartbeat, which if detected could be used to alert emergency personnel), and can be used to detect persons not otherwise moving.
In an embodiment, radar system 300 or a depth sensor 208 (as described below) when used in an ATM vestibule to detect loitering, may use a door status switch to reset the timer. Each time the door opens, for example when a new customer enters, the timer may be reset so that during the high traffic periods there is no alarm. Typically, people are loitering or sleeping in low traffic periods when the door does not usually open and close frequently.
Alternatively, instead of an ATM vestibule, radar system 300 could be used to detect the presence of a person in an ATM back room or a bank vault.
A depth map (or depth image) is an image that includes information relating to the distance of the surfaces of scene objects from a viewpoint such as from a depth sensor such as a 3D camera. For each pixel, or group of pixels, in the image of the depth map; there is associated a distance from the depth sensor. Depth maps can use a number of different means to show distance such as by luminance in proportion to the distance to the depth sensor, and by color. An example of luminance in proportion to the distance may be further distances darker and nearer distances lighter in a gray scale image, alternatively, it may be further distances lighter and nearer distances darker. An example of color depth map may use the red green blue (RGB) spectrum: red for further distances, yellow/green for middle distances, and blue for closer distances.
Depth sensors may use a number of different technologies to create depth maps. The technologies include Time-of-Flight (ToF), Stereo, and Structured Light.
Referring to
As shown in
For a number of applications, getting an approximate ‘size’ (or rough ‘volume’) of an object is sufficient. It may also be sufficient to just count the number of pixels above a certain height threshold which is closer to calculating the surface area of the object. Of course, once you have the surface area and the depth or height, the volume is easily calculated. For certain applications, only counting surfaces above a certain threshold is used to filter out “thin” objects that may be on the floor like a paper cup or piece of paper. Any “thick” objects above, for example, 4 inches is of interest.
The 3D cameras 1202 are angled slightly for a larger field of view; the ‘thickness’ difference calculation is still sufficient although the side of the furthest part of the object would now get occluded by the part of the object closer to the 3D cameras 1202. In the case of the pyramid with the base flat on the floor it would now appear to be elongated away from the 3D cameras 1202. If the angle is too steep, then the error may not be tolerable (say 90 degrees facing one of the side of the pyramid; the pyramid would appear to have the shape of a triangular prism). Thus, the 3D cameras 1202 using a relatively narrow field of view (or small viewing angles from the axis of the camera view) and not too steeply angled may achieve more reliable results.
Referring to
Referring to
Referring to
Referring to
Referring to
Referring to
There may be certain limitation with the 3D cameras 1202. The structured light 3D Cameras uses infrared (IR) light patterns to detect depth or distance to target. However, certain types of surfaces (reflective surfaces) reflect away the IR patterns of the structured light of 3D cameras, resulting in no reading (or zero depth) in the depth map. Further, when the ambient IR is strong, the IR patterns can be washed out, resulting in no readings as well. In all cases, in order to generate a stable and valid background model, the depth value of those “no reading” areas have to be estimated. The estimation is based on the neighbor pixels and is called interpolation. There are various methods of interpolation that could be used, for example, morphological filtering and bilinear filtering.
The generation of the model of the background 1904 also includes interpolating the height values (depth) for reflective regions where the 3D cameras 1202 is unable to detect the depth. The model of the background 1904 may be recalculated periodically. Once calculated, any new frames of the depth maps 1902 are subtracted from the model of the background 1904 to produce corresponding foreground frames 1906 (delta depth maps). The value of each pixel of the model of the background 1904 is subtracted from the value of each corresponding pixel of each frame of the depth maps 1902 to produce the foreground frames 1906 or delta depth maps. Where there is only one 3D camera, each depth map frame (a 3D camera shot) is compared to the model of the background to generate a corresponding foreground frame. The video analytics module 24 then analyzes the foreground frames 1906 to detect objects, large objects, and people loitering. The person 1810 is accordingly detected to be loitering after the person 1810 is detected to be in the foreground frames 1906 for a certain period of time and to be also sleeping on the floor if the maximum height is below a certain level, for example, 3 feet above the floor. The results are then displayed 1908 as shown in
Referring to
Referring to
In an alternative embodiment sensor 208, such as radar sensor 300 may be in a portable unit that can be carried by a single user. In this embodiment, sensor 208 may be powered by a battery or the like, and may include a display to notify users of the detection of a heartbeat or other indicates of a presence.
Alternatively, instead of ATM vestibules, sensor 208, when a radar system 300 or a depth sensor, may be used to determine loitering in other locations, such as an elevator.
Radar system 300 and depth sensors, as described above can have many other uses. For example radar system 300 and depth cameras 1202 may be used to monitor the presence of persons in a conference room or office, which can be useful for HVAC or lighting controls, by activating such systems only when persons are detected in the area. If radar system 300 is used, it can be positioned outside of a room to let a person outside by use of an indicator, such as a sign or light, know if the room is currently unoccupied and available, or if someone is in the room. Alternatively, radar system 300 directed out of a room, such as an office or meeting room, can let a person in the room know if a person is waiting outside without a disruption occurring. Similarly radar system 300 or depth cameras 1202 could be used to determine if a bathroom is occupied and alert a person outside.
Portable units with radar systems 300 could be used for search and rescue, military and police purposes. Radar systems 300 can be used to determine the location of a person needing assistance or posing a threat from outside a room or building.
Another use case occurs when determining if a person has been left behind, for example should an emergency occur in an office, or in a biohazard room, radar system 300 or a depth cameras 1202 can be used to determine if a person is still present in the affected area. In particular as radar signals pass through walls, radar system 300 can be positioned to determine if a person is within a room from a location outside the room. For example a radar system 300 can be positioned strategically to monitor a floor of an office building and be configured to quickly relay to rescue personnel whether a person is still breathing and where the person is on the floor.
The ability to detect whether a person is sleeping or not can be used in several different contexts. For example, hospital beds can be monitored to determine whether or not a patient is sleeping. Prison cells can be monitored to determine if a prisoner is awake or active when they should not be. Sleeping persons could be detected in alleys, subways or rail cars. Other uses would be elderly care rooms, bathrooms in hospitals or elderly care facilities, or even closets.
The fact that radar signals pass through walls allows radar system 300 to have a number of uses in multi-room environments to detect the presence or absence of persons in one or more rooms, and if a person is present, determine the length of time the person has been present. Radar system 300 can also be used to detect the level of activity of a person, for example if they are sleeping or moving.
One such multi room environment is a retail store changing room 2200, as shown in
The occupancy of the Rooms 01R-12R can be displayed to employees using a graph, as shown in
A time limit can be used to determine when the Room has been occupied longer than expected and may require assistance from employees. A colour, for example, red can be used to indicate to a user whether a Room has been occupied beyond a reasonable time. For example, as shown in
Another multi room environment in which radar system 300 and depth sensors are useful is prisons. As shown in
A report can be generated, an embodiment of which is partially shown in
A display can be presented to an authorized person, for example a prison guard, so that a quick glance to the display is enough to determine if anyone is not in a location in which they should be (e.g. missing from their prison cell, or out of a hospital room or care facility bed). Alarms or alerts can be generated should a prisoner (or patient) not be in the correct room. Besides prison cells, the system could be used with detention cells, holding cells, drunk tanks, and other areas where a person would be expected or not expected or it is desired to measure duration of a stay. The system could also be used in a farm environment, to allow farmers to ensure animals are present in their appropriate pens, and could be positioned in stables, tie-stalls or barns.
Another use of radar system 300 may be to detect the location of animals, such as mice or rats in a home. Small animals can be difficult to detect in a home and are often under floors or within walls. In this embodiment, radar system 300 may be portable to allow a user to move throughout a home to locate the animal, and use the ability of radar signals to pass through walls and floors to indicate the presence of a heartbeat of an animal.
Another use of radar system 300 or depth cameras 1202 could be used is waiting rooms, such as doctor offices, emergency rooms, and hotel or office lobbies to generate an alert if a person is detected. Restaurants, bars, and stock rooms could be monitored by radar system 300 or depth cameras 1202 to detect the presence of an unauthorized person.
Yet another use of radar system 300 is to monitor transportation facilities, such as a bus, train or taxi to ensure that no one is present.
Radar system 300 or depth cameras 1202
In the embodiments described above more than one presence detectors such as depth cameras 1202 or radar system 300 may be used.
Other uses for radar system 300 or a depth cameras 1202 may include people counting, or wrong way detection (for people or vehicles).
Yet another use for sensor 208 is to determine that a heartbeat is not present. For example, a user might believe they are seeing some sort of apparition, and could use sensor 208 to determine if the object being viewed has a heartbeat.
While the above description provides examples of the embodiments, it will be appreciated that some features and/or functions of the described embodiments are susceptible to modification without departing from the spirit and principles of operation of the described embodiments. Accordingly, what has been described above has been intended to be illustrated non-limiting and it will be understood by persons skilled in the art that other variants and modifications may be made without departing from the scope of the invention as defined in the claims appended hereto.
This application is a continuation of PCT International Application No. PCT/CA2017051179 filed on Oct. 3, 2017. This application also claims the benefit of U.S. Provisional Patent Application No. 62/404,124 filed on Oct. 4, 2016, and U.S. Provisional Patent Application No. 62/477,195 filed on Mar. 27, 2017. All of the above-mentioned applications are hereby incorporated by reference in their entireties.
| Number | Date | Country | |
|---|---|---|---|
| 62404124 | Oct 2016 | US | |
| 62477195 | Mar 2017 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/CA2017/051179 | Oct 2017 | US |
| Child | 15725218 | US |
