Patent Application
20240105036
This invention relates to subject(s) identification after human motion is detected in the sensing area through a device-free sensing approach.
Many currently used wireless communication systems such as LTE, LTE-Advance, IEEE 802.11n, IEEE 802.11ac (Wi-Fi 5), and IEEE 802.11ax (Wi-Fi 6) continuously sense the state of the wireless channel through well-known signals, or pilot signals, in order to dynamically optimize the transmission rate or improve the robustness of the system. These channel sensing mechanisms are continuously improving and enable self-driven calibration systems and wireless signal pre-compensation and post-compensation techniques, significantly improving the quality of wireless communication.
More fine-grained information is available in modern communication systems and several approaches have been proposed in order to improve these systems. For example, a method that provides periodic channel state information (CSI) data has been developed. However, these fine-grained measurements are not only valuable for controlling and optimizing communication networks and links as they can also be used for the purpose of detecting motion or human activities within a sensing area.
Several signals are broadcasted or emitted in type of frames by the stations (STA) and Access Points (APs) in Wi-Fi networks even without requiring association between them. For example, before two devices can associate to each other, each of them can read frames from the environment and each of them can decide to broadcast or send one or multiple frames or wireless signals in general.
Other aspects and features of the present invention will become apparent to those ordinarily skilled in the art upon review of the following description of specific embodiments of the invention in conjunction with the accompanying figures.
It is an object of the present invention to mitigate limitations within the prior art relating to subject(s) identification after human motion is detected in the sensing area through a device-free sensing approach.
In accordance with an embodiment of the invention there is provided a method comprising:
In accordance with an embodiment of the invention there is provided a method comprising:
In accordance with an embodiment of the invention there is provided a network comprising:
Other aspects and features of the present invention will become apparent to those ordinarily skilled in the art upon review of the following description of specific embodiments of the invention in conjunction with the accompanying figures.
Embodiments of the present invention will now be described, by way of example only, with reference to the attached Figures, wherein:
The present invention is directed to subject(s) identification after human motion is detected in the sensing area through a device-free sensing approach.
The ensuing description provides representative embodiment(s) only, and is not intended to limit the scope, applicability or configuration of the disclosure. Rather, the ensuing description of the embodiment(s) will provide those skilled in the art with an enabling description for implementing an embodiment or embodiments of the invention. It being understood that various changes can be made in the function and arrangement of elements without departing from the spirit and scope as set forth in the appended claims. Accordingly, an embodiment is an example or implementation of the inventions and not the sole implementation. Various appearances of “one embodiment,” “an embodiment” or “some embodiments” do not necessarily all refer to the same embodiments. Although various features of the invention may be described in the context of a single embodiment, the features may also be provided separately or in any suitable combination. Conversely, although the invention may be described herein in the context of separate embodiments for clarity, the invention can also be implemented in a single embodiment or any combination of embodiments.
Reference in the specification to “one embodiment”, “an embodiment”, “some embodiments” or “other embodiments” means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least one embodiment, but not necessarily all embodiments, of the inventions. The phraseology and terminology employed herein is not to be construed as limiting but is for descriptive purpose only. It is to be understood that where the claims or specification refer to “a” or “an” element, such reference is not to be construed as there being only one of that element. It is to be understood that where the specification states that a component feature, structure, or characteristic “may”, “might”, “can” or “could” be included, that particular component, feature, structure, or characteristic is not required to be included.
Reference to terms such as “left”, “right”, “top”, “bottom”, “front” and “back” are intended for use in respect to the orientation of the particular feature, structure, or element within the figures depicting embodiments of the invention. It would be evident that such directional terminology with respect to the actual use of a device has no specific meaning as the device can be employed in a multiplicity of orientations by the user or users.
Reference to terms “including”, “comprising”, “consisting” and grammatical variants thereof do not preclude the addition of one or more components, features, steps, integers or groups thereof and that the terms are not to be construed as specifying components, features, steps or integers. Likewise, the phrase “consisting essentially of”, and grammatical variants thereof, when used herein is not to be construed as excluding additional components, steps, features integers or groups thereof but rather that the additional features, integers, steps, components or groups thereof do not materially alter the basic and novel characteristics of the claimed composition, device or method. If the specification or claims refer to “an additional” element, that does not preclude there being more than one of the additional element.
A “portable electronic device” (PED) as used herein and throughout this disclosure, refers to a wireless device used for communications and other applications that requires a battery or other independent form of energy for power. This includes devices, but is not limited to, such as a cellular telephone, smartphone, personal digital assistant (PDA), portable computer, pager, portable multimedia player, portable gaming console, laptop computer, tablet computer, a wearable device and an electronic reader.
A “fixed electronic device” (FED) as used herein and throughout this disclosure, refers to a wireless and/or wired device used for communications and other applications that requires connection to a fixed interface to obtain power. This includes, but is not limited to, a laptop computer, a personal computer, a computer server, a kiosk, a gaming console, a digital set-top box, an analog set-top box, an Internet enabled appliance, an Internet enabled television, and a multimedia player.
A “wearable device” or “wearable sensor” relates to miniature electronic devices that are worn by the user including those under, within, with or on top of clothing and are part of a broader general class of wearable technology which includes “wearable computers” which in contrast are directed to general or special purpose information technologies and media development. Such wearable devices and/or wearable sensors may include, but not be limited to, smartphones, smart watches, e-textiles, smart shirts, activity trackers, smart glasses, environmental sensors, medical sensors, biological sensors, physiological sensors, chemical sensors, ambient environment sensors, position sensors, neurological sensors, drug delivery systems, medical testing and diagnosis devices, and motion sensors.
A “server” as used herein, and throughout this disclosure, refers to one or more physical computers co-located and/or geographically distributed running one or more services as a host to users of other computers, PEDs, FEDs, etc. to serve the client needs of these other users. This includes, but is not limited to, a database server, file server, mail server, print server, web server, gaming server, or virtual environment server.
An “application” (commonly referred to as an “app”) as used herein may refer to, but is not limited to, a “software application”, an element of a “software suite”, a computer program designed to allow an individual to perform an activity, a computer program designed to allow an electronic device to perform an activity, and a computer program designed to communicate with local and/or remote electronic devices. An application thus differs from an operating system (which runs a computer), a utility (which performs maintenance or general-purpose chores), and a programming tools (with which computer programs are created). Generally, within the following description with respect to embodiments of the invention an application is generally presented in respect of software permanently and/or temporarily installed upon a PED and/or FED.
A “subject” as used herein may refer to, but is not limited to, an individual or group of individuals. This includes, but is not limited to, private individuals, employees of organizations and/or enterprises, an unknown individual or an intruder, members of community organizations, members of charity organizations, men, women, and children. In its broadest sense the user may further include, but not be limited to, software systems, mechanical systems, robotic systems, android systems, etc. that may be characterized, i.e. identified, by one or more embodiments of the invention.
A “transmitter” (a common abbreviation for a radio transmitter or wireless transmitter) as used herein may refer to, but is not limited to, an electronic device which, with the aid of an antenna, produces radio waves. The transmitter itself generates a radio frequency alternating current containing the information to be transmitted which is applied to the antenna which radiates radio waves. A transmitter may be discrete, or it may form part of a transceiver in combination with a receiver. Transmitters may be employed within a variety of electronic devices that communicate by wireless signals including, but not limited to, PEDs, FEDs, wearable devices, two-way radios, and wireless beacons. A transmitter may operate according to one or more wireless protocols in dependence upon its design.
A “receiver” (a common abbreviation for a radio receiver or wireless receiver) as used herein may refer to, but is not limited to, an electronic device that receives radio waves via an antenna which converts them to a radio frequency alternating current wherein the receiver processes these signals to extract the transmitted information. Receivers may be employed within a variety of electronic devices that communicate by wireless signals including, but not limited to, PEDs, FEDs, wearable devices, two-way radios, and wireless beacons. A receiver may operate according to one or more wireless protocols in dependence upon its design.
A wireless transceiver comprises components needed for sending and receiving wireless signals, e.g. radiation system, amplifiers, filters, mixers, local oscillators, ADC and DAC, and any other component required in the modulator and demodulator.
“Device-free technology”, the target user(s) or the subject(s) do(es) not require to wear any device with him/her/them in order for the system or the technology to know that there is human motion in the sensing area or to detect the type of activities or not that the subject(s) are performing.
“Device-oriented technology”, the target is a device. The system or technology assumes, but not necessarily, that the subject(s) are wearing a device and what is tracked is the device.
A “wireless protocol” as used herein may refer to, but is not limited to, a specification defining the characteristics of a wireless network comprising transmitters and receivers such that the receivers can receive and convert the information transmitted by the transmitters. Such specifications may therefore define parameters relating to the wireless network, transmitters, and receivers including, but not limited to, frequency range, channel allocations, transmit power ranges, modulation format, error coding, etc. Such wireless protocols may include those agreed as national and/or international standards within those regions of the wireless spectrum that are licensed/regulated as well as those that are unlicensed such as the Industrial, Scientific, and Medical (ISM) radio bands and hence are met by equipment designed by a single original equipment manufacturer (OEM) or an OEM consortium. Such wireless protocols or wireless standards may include, but are not limited to, IEEE 802.11 Wireless LAN and any of their amendments, IEEE 802.16 WiMAX, GSM (Global System for Mobile Communications, IEEE 802.15 Wireless PAN, UMTS (Universal Mobile Telecommunication System), EV-DO (Evolution-Data Optimized), CDMA 2000, GPRS (General Packet Radio Service), EDGE (Enhanced Data Rates for GSM Evolution), Open Air, HomeRF, HiperLAN1/HiperLAN2, Bluetooth, ZigBee, Wireless USB, 6IoWPAN, and UWB (ultra-wideband).
A “wireless standard” as used herein and throughout this disclosure, refer to, but is not limited to, a standard for transmitting signals and/or data through electromagnetic radiation which may be optical, radio-frequency (RF) or microwave although typically RF wireless systems and techniques dominate. A wireless standard may be defined globally, nationally, or specific to an equipment manufacturer or set of equipment manufacturers. Dominant wireless standards at present include, but are not limited to IEEE 802.11, IEEE 802.15, IEEE 802.16, IEEE 802.20, UMTS, GSM 850, GSM 900, GSM 1800, GSM 1900, GPRS, ITU-R 5.138, ITU-R 5.150, ITU-R 5.280, IMT-1000, Bluetooth, Wi-Fi, Ultra-Wideband and WiMAX. Some standards may be a conglomeration of sub-standards such as IEEE 802.11 which may refer to, but is not limited to, IEEE 802.1a, IEEE 802.11b, IEEE 802.11g, or IEEE 802.11n as well as others under the IEEE 802.11 umbrella.
The system will collect as much information as possible of the device or devices that the subject is carrying on referred to as the target device or devices. Specific interaction (e.g. exchange of wireless signals) with the target device or devices is considered as well in order to maximize the data or information available for a future or immediate identification of the individual or subject. Any information collected during the observation period is used for identifying the individual or subject. The individual or person of interest may be a user of a product or an intruder.
A wireless device-free motion detection system according to an embodiment of the invention is illustrated in
The system is able to collect, through at least one of the devices in the network, which the transceivers 110 and 104 are connected to, a wide range of information from all or any of the devices (e.g. transceivers 110 and 104) within the area 100. As an example, this information includes but is not limited to Physical Layer (PHY layer), Media Access Control (MAC) sublayer and Logical Link Control (LLC) sublayer which are the two sublayers of the Data Link (DL) Layer of the OSI model. The PHY layer and the DL layer contain information about the frequency response of the channel, and/or phase response of the channel, and/or impulse response of the channel, and/or received signal strength indicators (RSSI), and/or the media access control address (MAC address) and/or, capture of Probe requests, capture of any broadcasting frame before the association between devices, control frames after or before association between devices, any frame related to the association process, and/or any other statistic that describes the wireless communication link between paired devices.
The system in
Now referring to
A portion or all of the analytics application 116 is hosted in a remote facility such as a cloud-based system, for example, such that at least one of Device 1 or Device 2 needs to be capable of connecting to the remote network upon which the Analytics Application 116 is hosted. If additional devices 204 are incorporated into the sensing system, the active sensing area 202 is enhanced and/or extended according to the number and location of new devices available within the communication network 200 and their wireless communication range. Enhancement of the sensing area occurs as a result of the increase in the number of data sources available. Extension of the sensing area also occurs as a result of the increase in overall reach of the wireless network 200. The scope of the systems and methods proposed herein are not limited by any particular network topology. The communication network 200 could be created by following any of the regulated communication standards, e.g. IEEE 802.11 standard family or some new standard. Further embodiments of the invention support structured networks as well as ad-hoc networks.
Any of the transceivers 110 and/or 104 in
Accordingly, the method proposed herein analyses flows of the information or data collected by any of the transceivers 110104 or Device 1, Device2, Device N as described above with respect to
According to other embodiments of the invention the system proposed herein can also create mechanisms for stimulating a target device or devices to be identified to keep transmitting signals by replying to a specific stimulus created by the system. For example, one of the transceivers 110 and/or 104 can create an SSID that is widely used in public spaces that offer free or not Wi-Fi services, e.g. free Wi-Fi, controlled access Wi-Fi through webpage (e.g. as employed in many retail environments) or paid Wi-Fi. Accordingly, the target device may get associated with the transceiver of the system according to an embodiment of the invention and expose its real MAC address instead of a random one. In the latter example, any other information that can serve to identify the target device will be collected and it is not limited only to the collection of a MAC address. For example, in the probe request there are multiple data fields that can be used as an input for an algorithm to identify the target device or devices later on.
The system proposed herein will use as many devices as available that can collect meaningful information for identifying the target device or devices when they are used somewhere else a posteriori or in-situs in the moment of an intrusion for example.
Accordingly, embodiments of the invention may include those implemented either in any of the devices forming the network, or in the cloud, or in a hybrid approach where some or all the devices in the network can partially compute, and/or cooperate with a cloud-based process 116 in the cloud system 118:
If in an intruder detection system or an area under surveillance motion is detected through a device-free sensing mechanism and an alarm, or alert, or flag variable, is set or goes off and a method as described below comprising:
Step 1: The transceivers 110 and/or 104 in the system proposed herein start identifying and/or collecting MAC addresses, probe requests and any of the information described above with respect to
Step 2: MAC addresses, probe requests and any of the information as described above with respect to
Step 3: If the MAC address is new, and/or there is no correlation between probe requests or any of the information, such as PHY, MAC, LLC, and DL layer information, for example, from the two comparisons A and B described above, then a Red Alarm or Red Alert is raised and MAC address(es), probe requests and any of the information, such as PHY, MAC, LLC, and DL layer information, for example, is recorded for future reference.
Step 4: If at least one MAC address is not new, and/or at least one device exhibits a correlation between its probe requests or any of the information, such as PHY, MAC, LLC, and DL layer information, for example, from the two comparison A and B described above, then an Orange Alert is raised and a call or a message to a known entity or entities is generated notifying the presence of a user of the system that was identified since they belong to a list of authorized device(s) with an specific MAC address, probe requests or any of the information, such as PHY, MAC, LLC, and DL layer information, for example, that identifies the device(s).
Step 5: If there are no probe requests to analyze or record, and none of the information, such as PHY, MAC, LLC, and DL layer information, for example, can help with the identification of the device or devices, such that there is no MAC address to analyze, then no identification of a potential device or devices is not possible, probably because the intruder has no wireless device with him/her, among other reasons, then an alarm is raised and/or an indication is set that subject (potential intruder) has not been identified.
In Step 1 regarding a potential interaction with the target device or target devices for extracting more information about them, then different stimulus or wireless signals can be sent from the system proposed herein. For example, the transceivers in the system can broadcast different well-known Wi-Fi network SSIDs and the MAC address of the target device or target devices might be revealed and captured by any of the transceivers proposed herein.
The information recorded in paragraph Step 3 can be any of the type of, and one element or subgroup of the information, such as PHY, MAC, LLC, and DL layer information, for example, can be used for identifying the target device or target devices and for the identification of the subject. The subject could be an intruder that is detected and/or identified with the systems and methods proposed herein.
Specific details are given in the above description to provide a thorough understanding of the embodiments. However, it is understood that the embodiments may be practiced without these specific details. For example, circuits may be shown in block diagrams in order not to obscure the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.
Implementation of the techniques, blocks, steps and means described above may be done in various ways. For example, these techniques, blocks, steps and means may be implemented in hardware, software, or a combination thereof. For a hardware implementation, the processing units may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described above and/or a combination thereof.
Also, it is noted that the embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process is terminated when its operations are completed, but could have additional steps not included in the figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination corresponds to a return of the function to the calling function or the main function.
Furthermore, embodiments may be implemented by hardware, software, scripting languages, firmware, middleware, microcode, hardware description languages and/or any combination thereof. When implemented in software, firmware, middleware, scripting language and/or microcode, the program code or code segments to perform the necessary tasks may be stored in a machine readable medium, such as a storage medium. A code segment or machine-executable instruction may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a script, a class, or any combination of instructions, data structures and/or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters and/or memory content. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.
For a firmware and/or software implementation, the methodologies may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. Any machine-readable medium tangibly embodying instructions may be used in implementing the methodologies described herein. For example, software codes may be stored in a memory. Memory may be implemented within the processor or external to the processor and may vary in implementation where the memory is employed in storing software codes for subsequent execution to that when the memory is employed in executing the software codes. As used herein the term “memory” refers to any type of long term, short term, volatile, nonvolatile, or other storage medium and is not to be limited to any particular type of memory or number of memories, or type of media upon which memory is stored.
Moreover, as disclosed herein, the term “storage medium” may represent one or more devices for storing data, including read only memory (ROM), random access memory (RAM), magnetic RAM, core memory, magnetic disk storage mediums, optical storage mediums, flash memory devices and/or other machine readable mediums for storing information. The term “machine-readable medium” includes, but is not limited to portable or fixed storage devices, optical storage devices, wireless channels and/or various other mediums capable of storing, containing or carrying instruction(s) and/or data.
The methodologies described herein are, in one or more embodiments, performable by a machine which includes one or more processors that accept code segments containing instructions. For any of the methods described herein, when the instructions are executed by the machine, the machine performs the method. Any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine are included. Thus, a typical machine may be exemplified by a typical processing system that includes one or more processors. Each processor may include one or more of a CPU, a graphics-processing unit, and a programmable DSP unit. The processing system further may include a memory subsystem including main RAM and/or a static RAM, and/or ROM. A bus subsystem may be included for communicating between the components. If the processing system requires a display, such a display may be included, e.g., a liquid crystal display (LCD). If manual data entry is required, the processing system also includes an input device such as one or more of an alphanumeric input unit such as a keyboard, a pointing control device such as a mouse, and so forth.
The memory includes machine-readable code segments (e.g. software or software code) including instructions for performing, when executed by the processing system, one of more of the methods described herein. The software may reside entirely in the memory, or may also reside, completely or at least partially, within the RAM and/or within the processor during execution thereof by the computer system. Thus, the memory and the processor also constitute a system comprising machine-readable code.
In alternative embodiments, the machine operates as a standalone device or may be connected, e.g., networked to other machines, in a networked deployment, the machine may operate in the capacity of a server or a client machine in server-client network environment, or as a peer machine in a peer-to-peer or distributed network environment. The machine may be, for example, a computer, a server, a cluster of servers, a cluster of computers, a web appliance, a distributed computing environment, a cloud computing environment, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. The term “machine” may also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.
The foregoing disclosure of the exemplary embodiments of the present invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many variations and modifications of the embodiments described herein will be apparent to one of ordinary skill in the art in light of the above disclosure. The scope of the invention is to be defined only by the claims appended hereto, and by their equivalents.
Further, in describing representative embodiments of the present invention, the specification may have presented the method and/or process of the present invention as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps set forth herein, the method or process should not be limited to the particular sequence of steps described. As one of ordinary skill in the art would appreciate, other sequences of steps may be possible. Therefore, the particular order of the steps set forth in the specification should not be construed as limitations on the claims. In addition, the claims directed to the method and/or process of the present invention should not be limited to the performance of their steps in the order written, and one skilled in the art can readily appreciate that the sequences may be varied and still remain within the spirit and scope of the present invention.
This application claims the benefit of priority as a continuation of U.S. patent application Ser. No. 17/748,365 filed May 19, 2022; which itself claims the benefit of priority as a continuation of U.S. patent application Ser. No. 17/199,952 filed Mar. 12, 2021 which has issued as U.S. Pat. No. 11,348,428; which itself claims the benefit of priority from U.S. Provisional Patent Application 62/988,846 filed Mar. 12, 2020; the entire contents of each being incorporated herein by reference.
| Number | Date | Country | |
|---|---|---|---|
| 62988846 | Mar 2020 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17748365 | May 2022 | US |
| Child | 18526656 | US | |
| Parent | 17199952 | Mar 2021 | US |
| Child | 17748365 | US |
