One or more embodiments described herein relate to non-contact, non-invasive sensing of vital signs or other health signals using radar or other high-resolution distance sensing.
Known radar systems transmit analog electromagnetic waves toward targets and receive echoes of the waves that reflect off the targets. Based on the distance between antennas that transmit the analog waves and the target objects, and/or movement of the target objects, the strength and/or frequency of the received echoes may change. The strength, frequency, and/or time-of-flight of the echoes may be used to derive the distance to the targets and/or movement of the targets.
Some known radar systems are limited in the accuracy at which the systems can measure distances to the targets. For example, the resolution at which these systems may be able to calculate the distance to targets may be relatively large. Moreover, some of these systems may have circuitry, such as a transmit/receive switch, that controls when the systems transmit waves or receive echoes. The switch can require a non-zero period of time to allow the systems to switch from transmitting waves to receiving echoes. This period of time may prevent the systems from being used to measure distances to targets that are relatively close, as the transmitted waves may reflect off the targets back to the receiving antennas before the systems can switch from transmission to reception. Additionally, some known systems have energy leakage from the transmitting antenna to the receiving antenna. This energy leakage can interfere with and/or obscure the measurement of distances to the targets and/or the detection of motion.
In an embodiment, a sensing system includes a first radar sensing assembly and an analysis system. The first radar sensing assembly measures plural distances to a first target location at different times using radar. The analysis system receives the plural distances from the first radar sensing assembly and quantifies movements of a target object at the first target location at the different times by calculating differences in the plural distances measured by the first radar sensing assembly. The analysis system generates one or more first quantified activity level values indicative of the movements of the target object at the first target location using the differences in the plural distances that are calculated.
In an embodiment, a sensing method includes transmitting first electromagnetic waves toward a first target location from a first radar sensing assembly at different times, receiving first echoes of the electromagnetic waves that are reflected off the first target location using the first radar sensing assembly, measuring plural distances to the first target location using the first echoes of the electromagnetic waves, quantifying movements of a target object at the first target location at the different times by calculating differences in the plural distances measured by the first radar sensing assembly, and generating one or more first quantified activity level values indicative of the movements of the target object at the first target location using the differences in the plural distances that are calculated.
The present subject matter will be better understood from reading the following description of non-limiting embodiments, with reference to the attached drawings, wherein below:
The accuracy of the distance or position sensing can be any level that allows for the extraction of meaningful health signals of the target, such as vital signs of the target, as described below. The sensing system may be radar-based that uses radar techniques to measure the distance and/or position to one or more targets. In one embodiment, the position and/or distance to the targets can be measured to less than 1 millimeter accuracy (e.g., 0.2 millimeter accuracy) at a relatively fast rate, such as 1000 times or more per second. Optionally, the sensing system may measure the position and/or distance at a different accuracy and/or at a different rate.
The sensing assembly may communicate (e.g., transmit and/or broadcast) data representative of the distance and/or location of the target to an analysis system 18 of the sensing system for storage and/or further analysis of the data. The data can be communicated through, over, and/or using one or more wired and/or wireless connections to the analysis system. The analysis system can include or represent one or more computer devices, such as one or more laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers).
The analysis system can extract or derive various other signals or measures of significance to the health or well-being of a person or animal (e.g., the target) from the data provided by the sensing system, as described below. For example, a signal related to the breathing of a patient or user can be extracted by the analysis system, or a signal related to the cardiac activity of the patient or user or the activity of the patient or user, or any other measure of the health or well-being of the patient or user may be derived. Additionally or alternatively, these health signals may be extracted or derived by the sensing system. These extracted signals may then be transmitted to another system for storage or further processing.
The sensing system collects high-resolution estimates or measurements of the position or distance to the one or more targets, which is referred to herein as the “raw” or “raw distance” or “raw position” data. This data can be communicated to the analysis system for storage and/or further processing. The raw data can be stored before or after the data is processed. The raw data can be processed or filtered, and/or other calculations can be performed on the raw data to extract or derive other signals, such as the respiration of the patient or user. These signals can be referred to as computed signals or auxiliary signals. The computed signals can also be stored in the analysis system or elsewhere. One or more auxiliary signals can be extracted, derived, or computed from the computed signal and then stored. In another embodiment, the raw data and all computed or extracted signals can be stored. Further signals can be extracted from the stored raw data in the future, and these can be saved as well. The raw data and extracted signals can all be used in any combination to compute other measures related to health or well-being. Alternatively, or in addition, the sensing system may possess sufficient computational ability to extract the signals and perform the computations. The sensing system may then transmit these signals to another system for storage or further analysis. It may transmit the raw data as well, or switch between two or more modes depending on other signals or observations. For example, the sensing system may extract a signal related to the presence of a subject (described below) and make an initial determination of the presence of a subject. Then, when it is determined that a subject is present, the sensing system switches modes to communicate the raw data to another computer system to perform further, more complex, analysis. When the data is transmitted to another system, the data may be further analyzed and/or stored.
The sensing system can continue to monitor the distance between the sensing assembly and the monitored location. When this distance or position changes or decreases (e.g., moves closer), the sensing system may infer that there is now a person or other subject (e.g., a target) 14 present in the monitored location, since the target has a non-zero thickness and the sensing assembly senses the absolute distance between the sensing system and the monitored location rather than simply monitoring motion. For example, the sensing assembly may not monitor motion, but instead may periodically, randomly, and/or upon operator demand measure the distance 22 to identify changes in the distance. Decreases in this distance (e.g., from the baseline distance 22 to a modified distance 30) may represent the arrival of a target at the monitored location. Conversely, increases in this distance can represent the departure of the target from the monitored location. The position or distance data that is acquired by the sensing system (e.g., the raw data) can be saved and/or filtered using a low-pass filter, a finite-impulse-response (FIR) filter or an infinite-impulse-response (IIR) filter, wavelet transform, or a combination of these techniques, or others. The presence or absence of the target from the monitored location can be used to assess various aspects of the health of the target, such as by monitoring how often and/or long the target remains in the monitored location.
The sensing system can measure different distances or position to a target that has entered the scene (e.g., the monitored location). Changes in the distances can be used to determine the position (e.g., within or not within the monitored location), posture (e.g., lying prone, sitting up, standing, or the like), or other relevant metric of the target. The position or distance data obtained by the sensing system (e.g. the raw data) can be saved and filtered using a low-pass filter, a finite-impulse-response (FIR) filter or an infinite impulse-response (IIR) filter, wavelet transform, or a combination of these techniques, or other filtering techniques.
While the sensing system is shown in
Additionally or alternatively, certain specific positions or distances of targets associated with various parts of a body of the target may indicate certain positions or postures of the target. Different measured distances between the sensing system and the target may be associated with different positions or postures of the target. For example, the sensing system can be mounted in the ceiling of a bedroom and directed toward a bed. A target measured at 5.5 feet (e.g., 168 centimeters) away from the sensing system can indicate that the target is lying in bed. A target measured at 4 feet (e.g., 122 centimeters) from the sensing system may indicate that the target is sitting in the bed. This can be used to assess various aspects of the health of the target. As another example, the sensing system can measure the distances between the sensing system and the monitored location and compare the distances and/or changes in the distances to one or more thresholds. These thresholds may be associated with different postures or positions of the target. For example, a distance between the sensing system and the monitored location that exceeds a first threshold may indicate that there is no target at the monitored location. A distance that exceeds a smaller, second threshold but not the first threshold (and/or a change in this distance that is smaller than a first differential threshold) may indicate that the target is lying down in the monitored location. A distance that exceeds a smaller, third threshold but not the second threshold (and/or a change in the distance that is smaller than a second differential threshold but larger than the first differential threshold) may indicate that the target is sitting up in the monitored location. A distance that exceeds a smaller, fourth threshold but not the third threshold (and/or a change in the distance that is smaller than a third differential threshold but larger than the second differential threshold) may indicate that the target is standing in the monitored location.
The position or posture of a person over time can be observed or measured as described above. This data can then be used to determine if the person suffered a fall during that time period. As an example, the sensing system may measure an object at feet (e.g., 168 centimeters) above a floor, which can correspond to the head of a person and can indicate that the person has entered a field of view of the sensing system. The systems may continue to monitor the distance to the person's body and observe that the body moved closer to the floor over a relatively short period of time. If the body then remains in relatively close proximity to the floor, for example less than two feet (e.g., 61 centimeters) or another distance, for an extended period of time (such as 30 seconds or another time period), the system may then produce an alert signal that indicates the person under observation has or had suffered a fall.
As another example, the sensing system may be directed toward a target, such as a person or animal. In the case of the target being a person or animal, the target location may be any part of the body, any external surface of the body, or an internal structure of the body. A new position or distance or change in the position or distance to a target from the sensing system can be indicative of physical activity of the target or target location. Information about the arrangement, positions, relative positions, locations, pointing directions, and/or other physical information about the physical environment of the targets and sensing system may be recorded. A time series or history of activity may be recorded over some period of time, such as seconds, minutes, hours, days, or even years or more. The time record of data can be used to extract a relative or absolute activity level for the target. The position or distance data (e.g., the raw data) can be saved and filtered using filtering, such as a low-pass filter, a finite-impulse-response (FIR) filter or an infinite-impulse-response (IIR) filter, wavelet transform, or a combination of these techniques, or others. If the physical environment information is used as well (e.g., if the monitoring location and/or locations of objects other than the target or target location are known), then the activity that is detected by the changes in distance as measured by the sensing system may be associated with activity in different parts of the body of the target. The situations of
The activity level may be quantified by the analysis system. For example, instead of merely identifying movement as an activity level, the analysis system may generate one or more numerical values indicative of the magnitude of the activity level. Larger activity level values indicate more movement of the target (e.g., larger movements and/or faster movements) while smaller activity level values indicate less movement of the target (e.g., smaller movements and/or slower movements). The quantified activity level values may be used to identify different types of movement events, as described herein. For example, relatively large activity level values may indicate that the target has fallen down, that the target is breathing rapidly, that the target has a relatively fast heart rate, and the like. Conversely, relatively small activity level values may indicate that the target is remaining still, that the target is breathing slowly or not at all, that the target has a relatively slow heart rate or no heart rate, and the like.
The position and/or distance from the sensing system to the target location is tracked or monitored over time. Changes in this distance and/or position can indicate breathing by the target. The analysis system 18 can monitor the distances and/or position, and/or changes in the distance and/or position, in order to identify and track respiration movements of the target. In order to avoid confusing respiration movements with other movements (e.g., rolling over in bed, shifting positions in bed, and the like), the analysis system may isolate the distances, positions, and/or changes in distances and/or positions representative of respiration movements from other distances, positions, and/or changes in distances and/or positions that are not representative of respiration movements. The analysis system may isolate the respiration-indicative distances, positions, and/or changes in distances and/or positions using one or more filters applied to the data acquired by the sensing system, such as by applying a low-pass filter, a finite-impulse response (FIR) filter or an infinite-impulse-response (IIR) filter, wavelet transform, a combination of filters, or one or more other filters. The analysis system or other system can monitor respiration of the target to assess various aspects of the health of the target. The analysis system can generate an output signal 82 that is communicated (e.g., transmitted and/or broadcast) to an output device 84, such as a monitor, speaker, haptic device, light, or the like. The output signal may direct the output device to generate an output, such as a visual, audible, and/or haptic alarm. For example, the analysis system can track breathing patterns during sleep for persons suffering from sleep apnea and generate the output signal to allow a physician to examine the breathing patterns of the target during sleep. As another example, the analysis system can track or monitor breathing of infants or others and generate an alarm by communicating the output signal to warn others of potential problems or cessation of breathing by the target, such as to prevent Sudden Infant Death Syndrome.
In one aspect of monitoring respiration, the analysis system may have information or collect information about the overall physical size of the chest of the target, such as via physical measurements, a camera, or other method. The analysis system can then combine this additional data with the distance or position data obtained by the sensing system. The change in distance or position of the front of the chest, combined with information about the lateral area or physical size of the chest can be used to calculate the volume of air that is taken into and then expelled from the lungs, such as the tidal volume.
The example of the sensing system and analysis system additionally or alternatively may be used to track or monitor cardiac functions of the target. The target location used by the sensing system can include the front of the chest, the rib cage, another structure around the heart, the heart, and/or another location that moves due to cardiac cycles of the target. This target location can be tracked or the distance or position between the sensing system and the target location measured, and can be indicative of cardiac functioning of the target. Similar to as described above with respect to monitoring respiration, however, some movements that are tracked by the sensing system may be from actions other than cardiac cycles. The position or distance data, e.g., the raw data, can be saved and/or filtered by the analysis system using one or more filters described herein in order to isolate the movements representative of cardiac functions from other movements. The analysis system can generate the output signal to the output device to allow the cardiac functions of the target to be analyzed.
In one aspect, the analysis system may simultaneously (or concurrently) use multiple levels or versions of the wavelet transform to examine the data generated by the sensing system. The analysis system can use a wavelet transform to extract a cardiac signal from raw position or distance information to the heart or the chest or other body structure, as measured and generated by the sensing system. Furthermore, multiple levels of the wavelet transform may be used by the analysis system to extract and/or reconstruct the cardiac signal. While a typical wavelet transform uses a single wavelet, such as the Symmlet-6 or the Daubechies-8 wavelet, the analysis system can process the raw distance, position, or motion data using more than one wavelet to extract the cardiac signal. Additionally, both of the above methods may be used together, for example, multiple levels of the Symmlet-7 and Daubechies-10 wavelets may be used in a wavelet transform to extract the cardiac signal.
The sensing system may have several target locations on the target and/or multiple sensing systems may be used, with two or more of the sensing systems having different target locations on the same target. For example, there may be multiple target locations in a radar beam generated by the sensing system or in a field of view of the sensing system. In an embodiment, the sensing system may be unable to distinguish between the multiple target locations. For example, the sensing system may be unable to separate the distance, position, and/or motion of the multiple target locations from each other. The sensing system can measure a superposition or summation of the positions and/or distances to the multiple target locations. The analysis system 18 may then process the data as described herein to extract or compute various observations of the position and/or distance or pattern of position and/or distance of the target locations.
The sensing system may be directed toward a target (e.g., a person or animal) that is standing or walking. The sensing system may be directed horizontally to measure the position and/or distance pattern or motion of the target as the target moves along a path. The analysis system may then extract or compute information regarding the gait or balance of the target from the distances and/or positions obtained by the sensing system. This can be used to assess various aspects of the health of the subject.
For example, the sensing system may sense the distance to one or more target locations of the target and the sensing system and/or analysis system may monitor the distances with respect to time to generate a time record of the positions and/or distances of one or more target locations. Such a record can include several different positions and/or distances, and the times at which the positions and/or distances were measured (e.g., time stamps). From this record, the motion, speed, velocity, and/or motion vector of the target and/or one or more target locations can be computed by the analysis system, such as by using the first difference method or another technique. Alternatively, a total change in position and/or distance to the target and/or one or more target locations may be computed by the analysis system, such as by a summation of the motion or another technique. If initial conditions or boundary conditions of the target are known, then the analysis system may compare the distances and/or locations sensed by the sensing system with these conditions in order to determine an actual position and/or distance of the target and/or one or more target locations. These boundary conditions may include the actual locations of surfaces around or near the target, such as locations and/or distances from the sensing system to walls, ceilings, floors, furniture, and the like. This computed motion or computed position may be used interchangeably in the various embodiments and examples of uses of the sensing system described herein.
The sensing system 4100 (and/or an analysis system described herein) can detect movement of the target along the linear segment 4106 and then the curved segment 4108 (or vice-versa) based on changes in the distance 4102. The distance 4102 may change when the target moves along the linear segment and then remain approximately the same when the target moves along the curved segment. For example, the sensing system 4100 may measure the distance 4102 as decreasing from the distance 4102A to the distance 41026, and then remaining the same (or substantially the same) from the distance 41026 to the distance 4102C. The sensing system 4100 (and/or the analysis system) can then determine that the target moved along the linear segment of the path and then the curved segment of the path due to these changes in the distances 4102.
The sensing systems 4100, 4200 (and/or an analysis system described herein) can detect movement of the target along the curved segment 4108 based on changes in the distances 4102 and/or 4202. The distances 4102 measured by the sensing system 4100 may not change when the target moves along the curved segment of the path, as described above. But, the distances 4202 measured by the other sensing system 4200 may change during this same movement. For example, the sensing system 4200 may measure the distance 4202 as increasing from the distance 4202A to the distance 4202B. The sensing systems 4100, 4200 (and/or the analysis system) can then determine that the target moved along the curved segment of the path based on these distances 4102, 4202.
Additionally or alternatively, the analysis system may use the time record to determine acceleration and/or or an acceleration vector of the target and/or one or more target locations, such as by the second difference method or another technique. Optionally, the sensing system and/or analysis system may determine and make a time record of the motion of the target and/or one or more target locations. The analysis system can use such a time record to compute an acceleration of the target and/or one or more target locations, such as by the first difference method or another technique. Alternatively, if the sensing system and/or analysis system generates data for a time record of acceleration of the target and/or one or more target locations, then a total change in motion and/or speed of the target and/or one or more target locations may be computed, such as by a summation of the acceleration or another technique. The analysis system may then compute a total position or distance change from the computed motion. Alternatively, the total change in position or distance may be calculated directly from the acceleration, such as by the second integral method or another technique. If initial conditions or boundary conditions of the target are known to the analysis system, then the analysis system may compute the actual velocity or position of the target and/or one or more target locations.
The analysis system (and/or sensing system) can calculate changes in the position data with respect to time in order to determine velocity data 5206 of the target. The velocity data also is shown alongside the horizontal and vertical axes. The magnitude of the values along the vertical axis for the velocity data may represent faster or slower moving speeds of the target. The analysis system (and/or sensing system) can calculate changes in the velocity data with respect to time in order to determine acceleration data 5208 of the target. The acceleration data also is shown alongside the horizontal and vertical axes. The magnitude of the values along the vertical axis for the acceleration data may represent faster or slower changes in the moving speed of the target.
Therefore, motion or velocity and acceleration may be computed from position or distance information obtained by the sensing system. Motion or velocity and position or distance may be computed from acceleration. For example, the analysis system can integrate the acceleration data with respect to time in order to obtain the moving speed of the target, and/or can integrate the velocity data with respect to time in order to determine the position or location of the target. Position or distance can be computed from velocity or speed and acceleration may be computed from velocity or speed. There are certain limitations on these computations, as described above, but it should be understood with all the methods and devices and systems described herein that the various sources or forms of information described in this section (position/distance, motion/speed, acceleration) can be used interchangeably where appropriate.
In one aspect, the sensing system and/or analysis system may be used in place of one or more other sensors that are used to monitor vital signs (or other information) about a target, such as a human patient in a hospital or other medical facility. As described herein, data representative of movement, respiration, cardiac functions, activity, falling, and the like, may be extracted from the data obtained by the sensing system. This data may be analyzed by the analysis system to generate vital sign signals that are similar or identical to data obtained using other types of sensors. These vital sign signals from the analysis system may represent the same information and/or be indistinguishable from vital sign signals or other signals collected by other types of sensors. Consequently, the vital sign or other signals obtained and/or generated by the sensing system and/or analysis system may then be used to replace other sensors used to monitor similar information or vital signs. In this way, the sensing system and/or analysis system may be used as one or more “virtual sensors” or “virtual sensor signals” that can be used in existing systems that had previously collected the information using other means. For example, in a hospital, a patient may need to wear multiple sensors to collect various vital sign or other signals, such as signals representative of cardiac cycles, respiration, and levels of activity. As described above, the sensing system and/or analysis system can be used in lieu of or in addition to the sensors worn by the patient. The sensing system and/or analysis system can produce virtual sensor signals that can be used by a hospital's information management system, with no further change to the system, to monitor the health of many patients.
Additional information (e.g., meta data) can be captured that relates to the physical environment and orientation of the sensing system. This information can include information about the arrangement, positions, relative positions, locations, pointing directions, orientation and other physical information about the physical environment of the targets and/or other objects in the vicinity of the target, and the sensing system may be recorded as well. This can include information about the sensing system itself as well as other objects in the physical vicinity, and boundaries of the environment, such as the walls. The information can also include information about the objects, such as materials and sizes. For example, the meta data could take a written form, such as “A sensing system is mounted on the ceiling, pointed straight down at the bed, where the person's torso would normally be. There is a table near the head of the bed.” The information can also be in a form usable by a computer or other logic or computation or information system. The information can also be recorded automatically, such as via one or more photographs of the environment around the sensing system. This information can be used to determine the association between a target location and another part of the target, or another object under observation. The analysis system may receive the data obtained by these other sensors and combine the data and/or compare the data with the data provided by the sensing system to determine such an association.
For example, multiple sensors (including the sensing system as a sensor) may be used in parallel to monitor and/or examine the same target. One or more of these sensors may have a limited field of view, such as in the case of a radar sensor of the sensing system, where a radar beam may only have a spread of seven degrees, or some other limited extent. Different sensors may be directed to different parts of the same target, such as by orienting multiple sensing systems toward different regions of the same target.
Instead of, or in addition to, sensing target locations that are part of the target, the sensing assembly 10 may measure distances to and/or positions of target locations of objects associated with, but not part of, a target. A target may be associated with various structures or parts, internal or external to the target, that make up the entire target. Additionally, there may be other indirect targets that have a distance or position that is related to the target, but are not part of the target. The sensing system can determine the distance, position, and/or motion of these indirect targets to extract information related to the target. For example, if a person sits in a chair and only the back of the chair is within the sensing range of the sensing system, then the sensing system and/or analysis system may sense the flex or other movement of the chair without actually sensing the person in order to determine the presence of the person in the chair. As another example, the sensing system may also sense the distance or position of blankets or clothing that is associated with a target.
Additionally or alternatively, the sensing system and/or analysis system may be used to identify individual persons and/or distinguish among persons. The analysis system can use wavelets or other signal processing techniques, pattern matching, artificial intelligence, or other methods to extract patterns of distance, position, and/or motion that are measured by the sensing system and associated with one or more target locations on of a person or animal. The parameters used to extract the vital signs for an individual person can also be used to identify the individual. For example, the analysis system can record one or more patterns of positions and/or distances or motion for a known individual person as obtained from one or more sensing systems. The one or more patterns may be unique to that individual person, or the probability of two or more individuals having matching or similar patterns may be so relatively small that the individuals can be identified using the patterns associated with the different individuals. Additionally or alternatively, the identification may be useful within a known population of individuals. The analysis system may include or have access to a set of patterns of distance and/or movement data associated with a set of individuals known to be at a given location (e.g., a nursing home). The analysis system may be unable to identify individuals outside of this set from distances and/or movement data obtained by the sensing system, but may be able to identify those individuals within the set. For example, while the identification may not be as unique as a fingerprint, but the identification may be good enough that the subject under observation is a certain person from the set of known possible subjects, for example a certain patient in a nursing home.
In one aspect, the sensing system can be hidden from view inside or behind another object. The sensing system (and associated antennas) can be designed to be tolerant of materials placed very close to the sensing system. Radio waves emitted by the sensing system (as described below) may be able to penetrate the material and allow the sensing system to operate properly, while remaining hidden from view for security, aesthetic, or other purposes. For example, a sensing system may be mounted behind a painting or other artwork hanging on a wall, with the sensing system mounted in a cavity behind the painting. As another example, the sensing system can be mounted in the ceiling of a home, with a front face of the sensing system flush with the surface of the ceiling. The sensing system could then be painted over to match the ceiling, thereby concealing the sensing system.
In the illustrated example, the sensing assembly 10 measures distance to the target or the position of a target. The target location may be the outer surface of the skin nearest to the sensing assembly 10, or somewhere else on the target, or another exposed part of the body, such as during surgery. The additional sensing system 7002 includes one or more light sources and/or optical sensors 7004, 7006. These sources and/or sensors can include light generating devices, such as lasers, that are directed toward the target to illuminate the target, such as the skin, at or near the same target location that the sensing assembly 10 is monitoring. The sources and/or sensors also can include one or more optical receivers that are sensitive to the same wavelengths of light as used in the light sources. The varying absorption and reflection of the light is measured by the optical receivers. The different magnitudes of absorption and reflection of the different wavelengths of light are affected by the amount of oxygen dissolved in the blood of the target, which may be used in a manner similar to reflective pulse oximetry.
The combination system described herein captures distance or position data at the same time as the light reflection and absorption data and can compute the amount of oxygen in the blood from a larger distance that is typically used for computing oxygen levels in blood, such as six feet (e.g., 1.8 meters) away or farther. The distance or position data obtained by the sensing assembly 10 can be used to correct or compensate for lower received optical power of the reflected light at the optical receivers due to the distance from the optical receivers to the patient or the body structure of the patient. For example, as the distance between the sensing system assembly 10 and a patient increases, a quantified amount of laser light that is reflected off the patient (e.g., a measured quantity representative of how much or how little of the laser light is received by a light sensor or meter after being reflected off the patient) may decrease due to the increased distance. Conversely, as this distance decreases, the amount of reflected light may increase. The analysis system can apply a correction factor to the measured amount of reflected light to correct for the impact of distance between the patient and the sensing system on this amount. For example, for larger distances, the analysis system can increase the amount of reflected light by an amount that is proportional to the distance. Additionally or alternatively, the analysis system can decrease the amount of reflected light by an amount that is proportional to the distance. Optionally, another technique may be used to correct the measured amount of reflected light. As a result, the corrected amount of reflected light may more accurately represent the amount of oxygen in the blood of the target object (e.g., patient).
For example, a raw data signal 6300 may be provided to the analysis system from the sensing system. The raw data signal can include data representative of positions of a target, such as distances between the sensing system and the chest of the target. The analysis system can calculate at 6302 (e.g., extract) the motion of the chest from this data, similar to as described above in connection with
The analysis system can remove (e.g., subtract) the breathing pattern signal 6304 from the raw data signal 6300 at 6306 in
Additionally or alternatively, one or more other signals of the target may be obtained from the raw data signal 6300. For example, after removing (e.g., filtering) the breathing pattern signal 6304, the cardiac signal 6310, and/or one or more other signals or portions of the raw data signal, an additional health signal 6312 may remain or be separately extracted at 6314 from the remainder of the raw data signal. This additional health data signal can represent one or more other vital signs of the target. One or more of the breathing pattern signal, the cardiac signal, and/or the additional health data signal may be output by the analysis system to one or more other systems that monitor the health or status of the target using the signal(s).
The techniques described herein can be used individually or together in any combination to enhance information collection, or gain a more comprehensive understanding of the subject, for example a person or animal. Alternatively, a subset of the techniques described herein may be used to collect or extract or compute a subset of the various signals described herein, as needed.
The techniques described herein can be used individually or in one or more combinations to extract the various signals described above, or others, including, but not limited to the presence, position or posture, activity, respiration and cardiac signals of a target, such as a person or animal. These extracted or computed signals can be used individually or in one or more combinations to assess the health of the target. The signals can be recorded over a period of time, including but not limited to, days, weeks, months, or years. These time records of the signals can then be used to extract more complex health measures or patterns. For example, a time record of data of one night while a target is sleeping can be used by the analysis system to compute the sleep health or other metrics related to the sleep of the target. If the sensing system continues to collect this nightly recording for a month (or other time period), then the analysis system can compute an overall assessment of the sleep patterns of the target. As another example, multiple sensing systems can be placed around the home of elderly or other at-risk person. The various signals from the sensing systems can be recorded over time. An initial time period can be used to establish a baseline or reference activity and vital sign pattern for the target. The sensing system and/or analysis system can then proceed with monitoring these signals for extended periods of time and look for deviations from the baseline patterns, which may indicate a change or decline in the health of the target. For example, the system can monitor these patterns over extended periods of time. If the patterns shift or decline over time then the health of the target is determined to be declining. The analysis system can notify a caregiver or other health care provider, such as a nurse, doctor, loved-one, or another system that steps should be taken to care for the subject.
The analysis system can automatically generate reports on the health or status of the target by examining the extracted or computed signals provided by the sensing system. Various algorithms may be applied to collected signals to make observations about the sleep, activities of daily living, or other health or wellness measures. The reports are then delivered to interested parties. The receiving parties may be the subject themselves to track their own health. The report recipient may also be a caregiver or doctor to track the health of the target.
The systems described herein may collect and transmit the raw distance, position, motion, and/or acceleration data of one or more target locations associated with a target, such as a person or animal. The sensing system and/or analysis system may communicate this data to another computational or electronic system for further processing, storage, and/or other purposes. The raw data may be saved for extended periods of time, or even indefinitely. At a later time, analysis that was not initially performed on the raw data may be performed. The reason that the analysis was not performed initially may be due to many factors, such as the user not paying for the analysis or the analysis simply not being necessary. Alternatively, new techniques may be developed that can use the same raw data, or new research or algorithms may emerge that can use the data. Then at any point in the future, a user or caregiver can request and/or pay for a retro-active analysis or screening of the data. Additionally another entity may request a new analysis of the data.
Alternatively or additionally, the outcomes and medical records of the subjects may be tracked or recorded. This health or medical information from one or more users can be compared to the raw data or the extracted or computed signals or vital signs or the extracted or computed health measures, such as sleep or activities of daily living. The long-term trends and patterns of all this data, across one or more users may be used to determine indications of emerging conditions in the users. For example, by comparing the data of subjects who were monitored and then had heart attacks it may come to pass that a certain breathing pattern is identified that indicates that the heart attack is imminent, such as in the next few days or weeks.
The raw distance or position data may be saved in the memory device for extended periods of time, or even indefinitely. One or more third-party systems or users 7006 may access the stored data, such as by using an application program interface (API) 7004. The services provided by the sensing and/or analysis system therefore can be expanded by a third-party. These third parties may access the data, signals, and/or measures described herein through the API and combine the data with other external information or know-how or methods or algorithms to perform other services for the user, subject or caregiver.
An output device 8004 generates one or more output signals to be communicated to other systems, persons, or the like, outside of the analysis system. For example, the control unit may generate cardiac signals, breathing pattern signals, and the like, and the output device may communicate these signals to systems outside of the analysis system. As another example, the output device may communicate data signals, an alert, or other information, to a display device in order to visually present the information obtained by the sensing system and/or examined by the analysis system.
In one aspect, the output device includes a display that visually presents any of this information or the extracted or computed signals or other health measures or vital signs to a user, or communicate the signals to another system for use elsewhere at approximately the same time. For example, the data and signals from multiple subjects in a hospital ward may all be sent to a display at a nurse's station so that they may continuously monitor the subjects or patients from a central location. The data and signals may also be saved for future use, as described above.
The sensing and/or analysis system may be used for additional applications as well. For example, the raw distance, position, motion, and/or acceleration data of one or more target locations associated with a target may be used to extract or compute signals or measures related to the health or medical condition of a subject, such as a person or animal. This data and/or the signals may be used to assess the sleep health or condition of a subject. It may be used to quantify the sleep stage of the subject, or their sleep/wake cycles, circadian rhythms, hypnogram, activity graph, or other measures of sleep. Additionally or alternatively, the data and/or the signals may be used to diagnose or screen for various sleep conditions or abnormalities such as sleep apnea, insomnia, or others.
When used to collect information about a person either with their knowledge or consent or not, the vital sign data or other data or signals can be used to determine if the person is lying or being untruthful. For example, certain breathing patterns, cardiac cycles, or other movements may be associated in the memory device with a person that is lying. The analysis system can compare the actual breathing patterns, cardiac cycles, or other movements to the known breathing patterns, cardiac cycles, or other movements that are associated with a lying person to determine if the person being monitored is lying.
As another example, the respiration of a patient or subject over time may be compared to pollen count data in their vicinity to determine if the person is allergic to various pollens, and which ones. As another example, the sleep health measure may be compared to a person's electronic calendar data to determine times or events that cause poor sleep.
As described herein, in an embodiment, a sensing system includes a first radar sensing assembly and an analysis system. The first radar sensing assembly measures plural distances to a first target location at different times using radar. The analysis system receives the plural distances from the first radar sensing assembly and quantifies movements of a target object at the first target location at the different times by calculating differences in the plural distances measured by the first radar sensing assembly. The analysis system generates one or more first quantified activity level values indicative of the movements of the target object at the first target location using the differences in the plural distances that are calculated. Optionally, the analysis system may use the distances instead of or in addition to using the differences in the distances. For example, instead of calculating changes in the distances that are measured to identify a posture, activity, fall, or other movement-related event of a person, the sensing system may measure an absolute distance to the person (e.g., the actual distance, such as 0.5 meters, 2 millimeters, 32.4562 centimeters, or other distance) and the analysis system can use this absolute distance to identify the movement or event of the person. The differences in distances may be referred to as relative distances, as the differences represent relative changes in the distances, but may not represent the absolute distances themselves.
In one aspect, the analysis system determines a posture of a human being using at least one of the plural distances or the differences in the plural distances that are measured by the radar sensing assembly. The posture includes the human at least one of standing, sitting, or lying on a surface.
In one aspect, the analysis system determines a presence of the target object at the common target location using at least one of the plural distances or the differences in the plural distances that are calculated.
In one aspect, the sensing system also includes a second radar sensing assembly measuring plural distances to a second target location at different times using radar. The second target location can be different than the first target location. The analysis system also quantifies movements of the target object at the second target location by calculating the differences in the plural distances measured by the second radar sensing assembly and generates one or more quantified second activity level values indicative of the movements of the target object at the second target location.
In one aspect, the first quantified activity level values represent the movements of a first body part of a human being and the second quantified activity level values represent the movements of a different, second body part of the same human being.
In one aspect, the analysis system detects a human being falling using the first quantified activity levels.
In one aspect, the first radar sensing assembly measures the plural distances to the first target location located on at least one of a front, side, or back of a chest of a human being and the analysis system determines a breathing pattern signal from the differences in the plural distances as the first quantified activity level values. The analysis system also may monitor respiration of the human being using the breathing pattern signal.
In one aspect, the analysis system extracts a cardiac signal from the breathing pattern signal as second quantified activity level values. The cardiac signal may represent cardiac cycles of the human being.
In one aspect, the analysis system extracts the cardiac signal as the health signal directly from the raw data signal without extracting the cardiac signal from another signal that is previously extracted from the raw data signal.
In one aspect, the target object is moving relative to the first radar sensing assembly, and the sensing system also includes a second radar sensing assembly measuring plural distances to the same first target location using radar. The analysis system tracks a movement path of the target object using the distances measured by the first radar sensing assembly and the second radar sensing assembly.
In one aspect, the analysis system at least one of stores or has access to a set of predetermined movement patterns associated with movements of plural different human beings and identifies a first human being as the target object based on the first quantified activity level values more closely matching a first predetermined movement pattern associated with the first human being than one or more other predetermined movement patterns in the set of the predetermined movement patterns.
In one aspect, the sensing system also includes one or more laser light sources generating laser light toward the first target location and one or more optical receivers detecting reflection of the laser light off the first target location. The analysis system correlates the first quantified activity level values with the reflection of the laser light in order to determine an oxygenation level of blood of the target object.
In an embodiment, a sensing method includes transmitting first electromagnetic waves toward a first target location from a first radar sensing assembly at different times, receiving first echoes of the electromagnetic waves that are reflected off the first target location using the first radar sensing assembly, measuring plural distances to the first target location using the first echoes of the electromagnetic waves, quantifying movements of a target object at the first target location at the different times by calculating differences in the plural distances measured by the first radar sensing assembly, and generating one or more first quantified activity level values indicative of the movements of the target object at the first target location using the differences in the plural distances that are calculated.
In one aspect, the sensing method also includes determining a posture of a human being using at least one of the plural distances or the differences in the plural distances that are measured by the radar sensing assembly. The posture includes the human at least one of standing, sitting, or lying on a surface.
In one aspect, the sensing method includes determining a presence of the target object at the common target location using at least one of the plural distances or the differences in the plural distances that are calculated.
In one aspect, the sensing method also includes transmitting second electromagnetic waves toward a second target location from a second radar sensing assembly. The second target location can be different than the first target location. The sensing method may further include receiving second echoes of the second electromagnetic waves that are reflected off the second target location using the second radar sensing assembly, measuring plural distances to the second target location using the second echoes of the electromagnetic waves, quantifying movements of the target object at the second target location at the different times by calculating differences in the plural distances measured by the second radar sensing assembly, quantifying movements of the target object at the second target location by calculating the differences in the plural distances measured by the second radar sensing assembly, and generating one or more quantified second activity level values indicative of the movements of the target object at the second target location.
In one aspect, the first quantified activity level values represent the movements of a first body part of a human being and the second quantified activity level values represent the movements of a different, second body part of the same human being.
In one aspect, the sensing method also includes detecting a human being falling using at least one of the first quantified activity levels or by tracking positions of one or more target locations on the target object over time.
In one aspect, the first radar sensing assembly measures the plural distances to the first target location located on a chest of a human being. The sensing method may also include determining a breathing pattern signal from the differences in the plural distances as the first quantified activity level values and monitoring respiration of the human being using the breathing pattern signal.
In one aspect, the sensing method also includes extracting a cardiac signal from the breathing pattern signal as second quantified activity level values. The cardiac signal represents cardiac cycles of the human being.
In one aspect, the method also includes extracting a health signal from the raw data signal, where the health signal includes at least one of a cardiac signal representative of cardiac cycles of the human being or another health signal.
In one aspect, the target object is moving relative to the first radar sensing assembly and the sensing method also includes transmitting second electromagnetic waves toward the same first target location from a second radar sensing assembly, receiving second echoes of the second electromagnetic waves that are reflected off the first target location using the second radar sensing assembly, measuring plural distances to the first target location using the second echoes of the electromagnetic waves, and tracking a movement path of the target object using the distances measured by the first radar sensing assembly and the second radar sensing assembly.
In one aspect, the sensing method also includes accessing a memory device that stores a set of predetermined movement patterns associated with movements of plural different human beings and identifying a first human being as the target object based on the first quantified activity level values more closely matching a first predetermined movement pattern associated with the first human being than one or more other predetermined movement patterns in the set of the predetermined movement patterns.
In one aspect, the sensing method also includes generating laser light toward the first target location from one or more laser light sources, detecting reflection of the laser light off the first target location, and correlating the first quantified activity level values with the reflection of the laser light in order to determine an oxygenation level of blood of the target object.
In one aspect, the sensing method also includes generating laser light toward the first target location from one or more laser light sources, detecting a quantified amount of reflection of the laser light off the first target location, and correcting the quantified amount of the reflection of the laser light using one or more of the plural distances that are measured in order to calculate an oxygenation level of blood in the target object.
The sensing assembly 102 can include a radar system that transmits electromagnetic pulse sequences as transmitted signals 106 toward the target object 104 that are at least partially reflected as echoes 108. Alternatively, the sensing assembly 102 can include an optical sensing system, such as a Light Detection And Ranging (LIDAR) system, that transmits light toward the target object 104 as the transmitted signals 106 and receives reflections of the light off the target object 104 as the echoes 108. In another embodiment, another method of transmission may be used, such as sonar, in order to transmit the transmitted signals 106 and receive the echoes 108.
A time of flight of the transmitted signals 106 and echoes 108 represents the time delay between transmission of the transmitted signals 106 and receipt of the echoes 108 off of the target object 104. The time of flight can be proportional to a distance between the sensing assembly 102 and the target object 104. The sensing assembly 102 can measure the time of flight of the transmitted signals 106 and echoes 108 and calculate a separation distance 110 between the sensing assembly 102 and the target object 104 based on the time of flight. The separation distance 110 that is measured or calculated can represent the distances between the target 14 and the sensing assembly 10 described above in one or more embodiments.
The sensing system 100 may include a control unit 112 (“External Control Unit” in
In one embodiment, the control unit 112 can be communicatively coupled with several sensing assemblies 102 located in the same or different places. For example, several sensing assemblies 102 that are remotely located from each other may be communicatively coupled with a common control unit 112. The control unit 112 can separately send control messages to each of the sensing assemblies 102 to individually activate (e.g., turn ON) or deactivate (e.g., turn OFF) the sensing assemblies 102. In one embodiment, the control unit 112 may direct the sensing assembly 102 to take periodic measurements of the separation distance 110 and then deactivate for an idle time to conserve power.
In one embodiment, the control unit 112 can direct the sensing assembly 102 to activate (e.g., turn ON) and/or deactivate (e.g., turn OFF) to transmit transmitted signals 106 and receive echoes 108 and/or to measure the separation distances 110. Alternatively, the control unit 112 may calculate the separation distance 110 based on the times of flight of the transmitted signals 106 and echoes 108 as measured by the sensing assembly 102 and communicated to the control unit 112. The control unit 112 can be communicatively coupled with an input device 114, such as a keyboard, electronic mouse, touchscreen, microphone, stylus, and the like, and/or an output device 116, such as a computer monitor, touchscreen (e.g., the same touchscreen as the input device 114), speaker, light, and the like. The input device 114 may receive input data from an operator, such as commands to activate or deactivate the sensing assembly 102. The output device 116 may present information to the operator, such as the separation distances 110 and/or times of flight of the transmitted signals 106 and echoes 108. The output device 116 may also connect to a communications network, such the internet.
The form factor of the sensing assembly 102 may have a wide variety of different shapes, depending on the application or use of the system 100. The sensing assembly 102 may be enclosed in a single enclosure 1602, such as an outer housing. The shape of the enclosure 1602 may depend on factors including, but not limited to, needs for power supply (e.g., batteries and/or other power connections), environmental protection, and/or other communications devices (e.g., network devices to transmit measurements or transmit/receive other communications). In the illustrated embodiment, the basic shape of the sensing assembly 102 is a rectangular box. The size of the sensing assembly 102 can be relatively small, such as three inches by six inches by two inches (7.6 centimeters by 15.2 centimeters by 5.1 centimeters), 70 mm by 140 mm by 10 mm, or another size. Alternatively, the sensing assembly 102 may have one or more other dimensions.
The sensing assembly 102 includes a front end 200 and a back end 202. The front end 200 may include the circuitry and/or other hardware that transmits the transmitted signals 106 and receives the reflected echoes 108. The back end 202 may include the circuitry and/or other hardware that forms the pulse sequences for the transmitted signals 106 or generates control signals that direct the front end 200 to form the pulse sequences for inclusion in the transmitted signals 106, and/or that processes (e.g., analyzes) the echoes 108 received by the front end 200. Both the front end 200 and the back end 202 may be included in a common housing. For example (and as described below), the front end 200 and the back end 202 may be relatively close to each other (e.g., within a few centimeters or meters) and/or contained in the same housing. Alternatively, the front end 200 may be remotely located from the back end 202. The components of the front end 200 and/or back end 202 are schematically shown as being connected by lines and/or arrows in
The front end 200 includes a transmitting antenna 204 and a receiving antenna 206. The transmitting antenna 204 transmits the transmitted signals 106 toward the target object 104 and the receiving antenna 206 receives the echoes 108 that are at least partially reflected by the target object 104. As one example, the transmitting antenna 204 may transmit radio frequency (RF) electromagnetic signals as the transmitted signals 106, such as RF signals having a frequency of 24 gigahertz (“GHz”) 1.5 GHz. Alternatively, the transmitting antenna 204 may transmit other types of signals, such as light, and/or at another frequency. In the case of light transmission the antenna may be replaced by a laser or LED or other device. The receiver may be replaced by a photo detector or photodiode.
A front end transmitter 208 (“RF Front-End,” “Transmitter, and/or “TX” in
An oscillating device 214 (“Oscillator” in
In the illustrated embodiment, the mixer 210A receives an in-phase (I) component or channel of a pattern signal 230A and mixes the I component or channel of the pattern signal 230A with the oscillating signal 216 to form an I component or channel of the transmitted signal 106. The mixer 210B receives a quadrature (Q) component or channel of a pattern signal 230B and mixes the I component or channel of the pattern signal 230B with the oscillating signal 216 to form a Q component or channel of the transmitted signal 106.
The transmitted signal 106 (e.g., one or both of the I and Q channels) is generated when the TX baseband signal 230 flows to the mixers 210. The digital output gate 250 may be disposed between the TX pattern generator and the mixers 210 for added control of the TX baseband signal 230. After a burst of one or more transmitted signals 106 is transmitted by the transmitting antenna 204, the sensing assembly 102 may switch from a transmit mode (e.g., that involves transmission of the transmitted signals 106) to a receive mode to receive the echoes 108 off the target object 104. In one embodiment, the sensing assembly 102 may not receive or sense the echoes 108 when in the transmit mode and/or may not transmit the transmitted signals 106 when in the receive mode. When the sensing assembly 102 switches from the transmit mode to the receive mode, the digital output gate 250 can reduce the amount of time that the transmit signal 106 generated by the transmitter 208 to the point that it is eliminated (e.g., reduced to zero strength). For example, the gate 250 can include tri-state functionality and a differential high-pass filter (which is represented by the gate 250). The baseband signal 230 passes through the filter before the baseband signal 230 reaches the upconversion mixer 210. The gate 250 can be communicatively coupled with, and controlled by, the control unit 112 (shown in
A front end receiver 218 (“RF Front-End,” “Receiver,” and/or “RX”) of the front end 200 is communicatively coupled with the receiving antenna 206. The front end receiver 218 receives an echo signal 224 representative of the echoes 108 (or data representative of the echoes 108) from the receiving antenna 206. The echo signal 224 may be an analog signal in one embodiment. The receiving antenna 206 may generate the echo signal 224 based on the received echoes 108. In the illustrated embodiment, an amplifier 238 may be disposed between the receive antenna 206 and the front end receiver 218. The front end receiver 218 can include an amplifier 220 and mixers 222A, 222B. Alternatively, one or more of the amplifiers 220, 238 may not be provided. The amplifiers 220, 238 can increase the strength (e.g., gain) of the echo signal 224. The mixers 222A, 222B may include or represent one or more mixing devices that receive different components or channels of the echo signal 224 to mix with the oscillating signal 216 (or a copy of the oscillating signal 216) from the oscillating device 214. For example, the mixer 222A can combine the analog echo signal 224 and the I component of the oscillating signal 216 to extract the I component of the echo signal 224 into a first baseband echo signal 226A that is communicated to the back end 202 of the sensing assembly 102. The first baseband echo signal 226A may include the I component or channel of the baseband echo signal. The mixer 222B can combine the analog echo signal 224 and the Q component of the oscillating signal 216 to extract the Q component of the analog echo signal 224 into a second baseband echo signal 226B that is communicated to the back end 202 of the sensing assembly 102. The second baseband echo signal 226B can include the Q component or channel of the baseband echo signal. In one embodiment, the echo signals 226A, 226B can be collectively referred to as a baseband echo signal 226. In one embodiment, the mixers 222A, 222B can multiply the echo signal 224 by the I and Q components of the oscillating signal 216 to form the baseband echo signals 226A, 226B.
The back end 202 of the sensing assembly 102 includes a transmit (TX) pattern code generator 228 that generates the pattern signal 230 for inclusion in the transmitted signal 106. The transmit pattern code generator 228 includes the transmit code generators 228A, 228B. In the illustrated embodiment, the transmit code generator 228A generates the I component or channel pattern signal 230A (“I TX Pattern” in
The transmit pattern code generator 228 creates the pattern of bits and communicates the pattern in the pattern signals 230A, 230B to the front end transmitter 208. The pattern signals 230A, 230B may be individually or collectively referred to as a pattern signal 230. In one embodiment, the pattern signal 230 may be communicated to the front end transmitter 208 at a frequency that is no greater than 3 GHz. Alternatively, the pattern signal 230 may be communicated to the front end transmitter 208 at a greater frequency. The transmit pattern code generator 228 also communicates the pattern signal 230 to a correlator device 232 (“Correlator” in
The backend section 202 includes or represents hardware (e.g., one or more processors, controllers, and the like) and/or logic of the hardware (e.g., one or more sets of instructions for directing operations of the hardware that is stored on a tangible and non-transitory computer readable storage medium, such as computer software stored on a computer memory). The RX backend section 202B receives the pattern signal 230 from the pattern code generator 228 and the baseband echo signal 226 (e.g., one or more of the signals 226A, 226B) from the front end receiver 200. The RX backend section 202B may perform one or more stages of analysis of the baseband echo signal 226 in order to determine the separation distance 110 and/or to track and/or detect movement of the target object 104.
The stages of analysis can include a coarse stage, a fine stage, and/or an ultrafine stage, as described above. In the coarse stage, the baseband processor 232 compares the pattern signal 230 with the baseband echo signal 226 to determine a coarse or estimated time of flight of the transmitted signals 106 and the echoes 108. For example, the baseband processor 232 can measure a time delay of interest between the time when a transmitted signal 106 is transmitted and a subsequent time when the pattern in the pattern signal 230 (or a portion thereof) and the baseband echo signal 226 match or substantially match each other, as described below. The time delay of interest may be used as an estimate of the time of flight of the transmitted signal 106 and corresponding echo 108.
In the fine stage, the sensing assembly 102 can compare a replicated copy of the pattern signal 230 with the baseband echo signal 226. The replicated copy of the pattern signal 230 may be a signal that includes the pattern signal 230 delayed by the time delay of interest measured during the coarse stage. The sensing assembly 102 compares the replicated copy of the pattern signal 230 with the baseband echo signal 226 to determine a temporal amount or degree of overlap or mismatch between the replicated pattern signal and the baseband echo signal 226. This temporal overlap or mismatch can represent an additional portion of the time of flight that can be added to the time of flight calculated from the coarse stage. In one embodiment, the fine stage examines I and/or Q components of the baseband echo signal 226 and the replicated pattern signal.
In the ultrafine stage, the sensing assembly 102 also can examine the I and/or Q component of the baseband echo signal 226 and the replicated pattern signal to determine a temporal overlap or mismatch between the I and/or Q components of the baseband echo signal 226 and the replicated pattern signal. The temporal overlap or mismatch of the Q components of the baseband echo signal 226 and the replicated pattern signal may represent an additional time delay that can be added to the time of flight calculated from the coarse stage and the fine stage (e.g., by examining the I and/or Q components) to determine a relatively accurate estimation of the time of flight. Alternatively or additionally, the ultrafine stage may be used to precisely track and/or detect movement of the target object 104 within the bit of interest. The terms “fine” and “ultrafine” are used to mean that the fine stage may provide a more accurate and/or precise (e.g., greater resolution) calculation of the time of flight (tF) and/or the separation distance 110 relative to the coarse stage and that the ultrafine stage may provide a more accurate and/or precise (e.g., greater resolution) calculation of the time of flight (tF) and/or the separation distance 110 relative to the fine stage and the coarse stage. Alternatively or additionally, the time lag of the waveforms in the I channel and Q channel can be examined to resolve phases of the echoes in order to calculate separation distance or motion of the target.
As described above, the ultrafine stage determination may involve a similar process as the coarse stage determination. For example, the coarse stage determination may examine the I channel of the receive pattern and the data stream to determine correlation values of different subsets of the data stream and, from those correlation values, determine a subset of interest and a corresponding time-of-flight, as described herein. The ultrafine stage determination can use the I and/or Q channel of the receive pattern and the data stream to determine correlation values of different subsets of the data stream and, from those correlation values, determine a subset of interest and a time-of-flight. The times-of-flight from the I channel and Q channel can be combined (e.g., averaged) to calculate a time of flight and/or separation distance to the target. The correlation values calculated by the ultrafine stage determination can be used to calculate an additional time delay that can be added to the time delays from the coarse stage and/or the fine stage to determine a time of flight and/or separation distance to the target. Alternatively or additionally, the correlation values of the waveforms in the I channel and Q channel can be examined to resolve phases of the echoes in order to calculate separation distance or motion of the target.
The backend 202 can include a first baseband processor 232A (“I Baseband Processor” in
As described below, a correlation window that also includes the pattern (e.g., the pulse sequence of bits) or a portion thereof that was transmitted in the transmitted signal 106 may be compared to the baseband echo signal 226. The correlation window may be progressively shifted or delayed from a location in the baseband echo signal 226 representative of a start of the echo signal 226 (e.g., a time that corresponds to the time at which the transmitted signal 106 is transmitted, but which may or may not be the exact beginning of the baseband echo signal) and successively, or in any other order, compared to different subsets or portions of the baseband echo signal 226. Correlation values representative of degrees of match between the pulse sequence in the correlation window and the subsets or portions of the baseband echo signal 226 can be calculated and a time delay of interest (e.g., approximately the time of flight) can be determined based on the time difference between the start of the baseband echo signal 226 and one or more maximum or relatively large correlation values. The maximum or relatively large correlation value may represent at least partial reflection of the transmitted signals 106 off the target object 104, and may be referred to as a correlation value of interest.
As used herein, the terms “maximum,” “minimum,” and forms thereof, are not limited to absolute largest and smallest values, respectively. For example, while a “maximum” correlation value can include the largest possible correlation value, the “maximum” correlation value also can include a correlation value that is larger than one or more other correlation values, but is not necessarily the largest possible correlation value that can be obtained. Similarly, while a “minimum” correlation value can include the smallest possible correlation value, the “minimum” correlation value also can include a correlation value that is smaller than one or more other correlation values, but is not necessarily the smallest possible correlation value that can be obtained.
The time delay of interest can then be used to calculate the separation distance 110 from the coarse stage. For example, in one embodiment, the separation distance 110 may be estimated or calculated as:
where d represents the separation distance 110, tF represents the time delay of interest (calculated from the start of the baseband echo signal 226 to the identification of the correlation value of interest), and c represents the speed of light. Alternatively, c may represent the speed at which the transmitted signals 106 and/or echoes 108 move through the medium or media between the sensing assembly 102 and the target object 104. In another embodiment, the value of tF and/or c may be modified by a calibration factor or other factor in order to account for portions of the delay between transmission of the transmitted signals 106 and receipt of the echoes 108 that are not due to the time of flight of the transmitted signals 106 and/or echoes 108.
With continued reference to the sensing assembly 102 shown in
The baseband echo signal 226 includes in one embodiment a sequence of square waves (e.g., low and high values 328, 330), but the waves may have other shapes. The echo signal 226 may be represented as a digital echo signal 740 (shown and described below in connection with
The baseband echo signal 226 begins at a transmission time (t0) of the axis 304. The transmission time (t0) may correspond to the time at which the transmitted signal 106 is transmitted by the sensing assembly 102. Alternatively, the transmission time (t0) may be another time that occurs prior to or after the time at which the transmitted signal 106 is transmitted.
The baseband processor 232 obtains a receive pattern signal 240 from the pattern generator 228, similar to the transmit pattern (e.g., in the signal 230) that is included in the transmitted signal 106, the receive pattern signal 240 may include a waveform signal representing a sequence of bits, such as a digital pulse sequence receive pattern 306 shown in
The baseband processor 232 uses all or a portion of the receive pattern 306 as a correlation window 320 that is compared to different portions of the digitized echo signal 740 in order to calculate correlation values (“CV”) at the different positions. The correlation values represent different degrees of match between the receive pattern 306 and the digitized echo signal 740 across different subsets of the bits in the digitized echo signal 740. In the example illustrated in
For example, the correlator device 731 may compare the bits in the correlation window 320 to a first subset 308 of the bits 300, 302 in the digitized echo signal 740. For example, the correlator device 731 can compare the receive pattern 306 with the first six bits 300, 302 of the digitized echo signal 740. Alternatively, the correlator device 731 can begin by comparing the receive pattern 306 with a different subset of the digitized echo signal 740. The correlator device 731 calculates a first correlation value for the first subset 308 of bits in the digitized echo signal 740 by determining how closely the sequence of bits 300, 302 in the first subset 308 match the sequence of bits 300, 302 in the receive pattern 306.
In one embodiment, the correlator device 731 assigns a first value (e.g., +1) to those bits 300, 302 in the subset of the digitized echo signal 740 being compared to the correlation window 320 that match the sequence of bits 300, 302 in the correlation window 320 and a different, second value (e.g., −1) to those bits 300, 302 in the subset of the digitized echo signal 740 being examined that do not match the sequence of bits 300, 302 in the correlation window 320. Alternatively, other values may be used. The correlator device 731 may then sum these assigned values for the subset of the digitized echo signal 740 to derive a correlation value for the subset.
With respect to the first subset 308 of bits in the digitized echo signal, only the fourth bit (e.g., zero) and the fifth bit (e.g., one) match the fourth bit and the fifth bit in the correlation window 320. The remaining four bits in the first subset 308 do not match the corresponding bits in the correlation window 320. As a result, if +1 is assigned to the matching bits and −1 is assigned to the mismatching bits, then the correlation value for the first subset 308 of the digitized echo signal 740 is calculated to be −2. On the other hand, if +1 is assigned to the bits and 0 is assigned to the mismatching bits, then the correlation value for the first subset 308 of the digitized echo signal 740 is calculated to be +2. As described above, other values may be used instead of +1 and/or −1.
The correlator device 731 then shifts the correlation window 320 by comparing the sequence of bits 300, 302 in the correlation window 320 to another (e.g., later or subsequent) subset of the digitized echo signal 740. In the illustrated embodiment, the correlator device 731 compares the correlation window 320 to the sixth through seventh bits 300, 302 in the digitized echo signal 740 to calculate another correlation value. As shown in
The correlator device 731 may continue to compare the correlation window 320 to different subsets of the digitized echo signal 740 to calculate correlation values for the subsets. In continuing with the above example, the correlator device 731 calculates the correlation values shown in
In another embodiment, the receive pattern 306 that is included in the correlation window 320 and that is compared to the subsets of the digitized echo signal 740 may include a portion, and less than the entirety, of the transmit pattern that is included in the transmitted signal 106 (shown in
In one embodiment, the correlator device 731 can compare less than the entire receive pattern 306 to the subsets by applying a mask to the receive pattern 306 to form the correlation window 320 (also referred to as a masked receive pattern). With respect to the receive pattern 306 shown in
The correlator 731 may identify a correlation value that is largest, that is larger than one or more correlation values, and/or that is larger than a designated threshold as a correlation value of interest 312. In the illustrated example, the fifth correlation value (e.g., +6) may be the correlation value of interest 312. The subset or subsets of bits in the digitized echo signal 740 that correspond to the correlation value of interest 312 may be identified as the subset or subsets of interest 314. In the illustrated example, the subset of interest 314 includes the fifth through tenth bits 300, 302 in the digitized echo signal 740. In this example, if the start of the subset of interest is used to identify the subset of interest then the delay of interest would be five. Multiple subsets of interest may be identified where the transmitted signals 106 (shown in
Each of the subsets of the digitized echo signal 740 may be associated with a time delay (td) between the start of the digitized echo signal 740 (e.g., t0) and the beginning of the first bit in each subset of the digitized echo signal 740. Alternatively, the beginning of the time delay (td) for the subset can be measured from another starting time (e.g., a time before or after the start of the digitized echo signal 740 (t0) and/or the end of the time delay (td) may be at another location in the subset, such as the middle or at another bit of the subset.
The time delay (td) associated with the subset of interest may represent the time of flight (tF) of the transmitted signal 106 that is reflected off a target object 104. Using Equation #1 above, the time of flight can be used to calculate the separation distance 110 between the sensing assembly 102 and the target object 104. In one embodiment, the time of flight (tF) may be based on a modified time delay (td), such as a time delay that is modified by a calibration factor to obtain the time of flight (tF). As one example, the time of flight (tF) can be corrected to account for propagation of signals and/or other processing or analysis. Propagation of the echo signal 224, formation of the baseband echo signal 226, propagation of the baseband echo signal 226, and the like, through the components of the sensing assembly 102 can impact the calculation of the time of flight (tF). The time delay associated with a subset of interest in the baseband echo signal 226 may include the time of flight of the transmitted signals 106 and echoes 108, and also may include the time of propagation of various signals in the analog and digital blocks (e.g., the correlator device 731 and/or the pattern code generator 228 and/or the mixers 210 and/or the amplifier 238) of the system 100.
In order to determine the propagation time of data and signals through these components, a calibration routine can be employed. A measurement can be made to a target of known distance. For example, one or more transmitted signals 106 can be sent to the target object 104 that is at a known separation distance 110 from the transmit and/or receiving antennas 204, 206. The calculation of the time of flight for the transmitted signals 106 can be made as described above, and the time of flight can be used to determine a calculated separation distance 110. Based on the difference between the actual, known separation distance 110 and the calculated separation distance 110, a measurement error that is based on the propagation time through the components of the sensing assembly 102 may be calculated. This propagation time may then be used to correct (e.g., shorten) further times of flight that are calculated using the sensing assembly 102.
In one embodiment, the sensing assembly 102 may transmit several bursts of the transmitted signal 106 and the correlator device 731 may calculate several correlation values for the digitized echo signals 740 that are based on the reflected echoes 108 of the transmitted signals 106. The correlation values for the several transmitted signals 106 may be grouped by common time delays (td), such as by calculating the average, median, or other statistical measure of the correlation values calculated for the same or approximately the same time delays (td). The grouped correlation values that are larger than other correlation values or that are the largest may be used to more accurately calculate the time of flight (tF) and separation distance 110 relative to using only a single correlation value and/or burst.
As described above, the received echo signal 224 may be conditioned by circuits 506 (e.g., by the front end receiver 218 shown in
Also as described above, the pattern code generator 228 generates the pattern (e.g., a digital pulse sequence) that is communicated in the pattern signal 230. The digital pulse sequence may be relatively high speed in order to make the pulses shorter and increase accuracy and/or precision of the system 100 (shown in
In one embodiment, the digital pulse sequence is generated by one or more digital circuits, such as a relatively low-power Field-Programmable Gate Array (FPGA) 504. The FPGA 504 may be an integrated circuit designed to be configured by the customer or designer after manufacturing to implement a digital or logical system. As shown in
The common frequency reference generator 604 may be or include the oscillator device 214 shown in
In one embodiment, the reference generator 604 emits a frequency reference signal 216 that is a sinusoidal wave at one half the frequency of the carrier frequency. The reference signal is split equally and delivered to the transmitter 600 and the receiver 602. Although the reference generator 604 may be able to vary the frequency of the reference signal 216 according to an input control voltage, the reference generator 604 can be operated at a fixed control voltage in order to cause the reference generator 604 to output a fixed frequency reference signal 216. This is acceptable since frequency coherence between the transmitter 600 and the receiver 602 may be automatically maintained. Furthermore, this arrangement can allow for coherence between the transmitter 600 and the receiver 602 without the need for a phase locked loop (PLL) or other control structure that may limit the accuracy and/or speed at which the sensing assembly 102 operates. In another embodiment a PLL may be added to for other purposes, such as stabilizing the carrier frequency or otherwise controlling the carrier frequency.
The reference signal 216 can be split and sent to the transmitter 600 and receiver 602. The reference signal 216 drives the transmitter 600 and receiver 602, as described above. The transmitter 600 may drive (e.g., activate to transmit the transmitted signal 106 shown in
In one embodiment, the transmitter 600 can take separate in-phase (I) and quadrature (Q) digital patterns or signals from the pattern generator 604 and/or the pattern code generator 228 (shown in
As described above, the receiver 602 may also receive a copy of the frequency reference signal 216 from the reference generator 604. The returning echoes 108 (shown in
The receiver 602 can down-convert a relatively wide block of frequency spectrum centered on the carrier frequency to produce the baseband signal (e.g., the baseband echo signal 226 shown in
The frequency reference signal 216 may contain or comprise two or more individual signals such as the I and Q components that are phase shifted relative to each other. The phase shifted signals can also be generated internally by the transmitter 600 and the receiver 602. For example, the signal 216 may be generated to include two or more phase shifted components (e.g., I and Q components or channels), or may be generated and later modified to include the two or more phase shifted components.
In one embodiment, the front end 200 provides relatively high isolation between the transmit signal 606 and the echo signal 224. This isolation can be achieved in one or more ways. First, the transmit and receive components (e.g., the transmitter 600 and receiver 602) can be disposed in physically separate chips, circuitry, or other hardware. Second, the reference generator 604 can operate at one half the carrier frequency so that feed-through can be reduced. Third, the transmitter 600 and the receiver 602 can have dedicated (e.g., separate) antennas 204, 206 that are also physically isolated from each other. This isolation can allow for the elimination of a TX/RX switch that may otherwise be included in the system 100. Avoiding the use of the TX/RX switch also can remove the switch-over time between the transmitting of the transmitted signals 106 and the receipt of the echoes 108 shown in
In one embodiment, the system 100 (shown in
The baseband processing system 232 receives the echo signal 226 (e.g., the I component or channel of the echo signal 226A and/or the Q component or channel of the echo signal 226B from the front end receiver 218 (shown in
The digitized echo signal 226 that is received by the system 232 may be conditioned by signal conditioning components of the baseband processing system 232, such as by modifying the signals using a conversion amplifier 704 (e.g., an amplifier that converts the baseband echo signal 226, such as by converting current into a voltage signal). In one embodiment, the conversion amplifier 704 includes or represents a trans-impedance amplifier, or “TIA” in
The second amplifier 706 may be used to determine the sign of the input differential signal 708 and the times at which the sign changes from one value to another. For example, the second amplifier 706 may act as an analog-to-digital converter with only one bit precision in one embodiment. Alternatively, the second amplifier 706 may be a high-speed analog-to-digital converter that periodically samples the differential signal 708 at a relatively fast rate. Alternatively, the second amplifier may act as an amplitude quantizer while preserving timing information of the baseband signal 226. The use of a limiting amplifier as the second amplifier 706 can provide relatively high gain and relatively large input dynamic range. As a result, relatively small differential signals 708 that are supplied to the limiting amplifier can result in a healthy (e.g., relatively high amplitude and/or signal-to-noise ratio) output signal 710. Additionally, larger differential signals 708 (e.g., having relatively high amplitudes and/or energies) that may otherwise result in another amplifier being overdriven instead result in a controlled output condition (e.g., the limiting operation of the limiting amplifier). The second amplifier 706 may have a relatively fast or no recovery time, such that the second amplifier 706 may not go into an error or saturated state and may continue to respond to the differential signals 708 that are input into the second amplifier 706. When the input differential signal 708 returns to an acceptable level (e.g., lower amplitude and/or energy), the second amplifier 706 may avoid the time required by other amplifiers for recovery from an overdrive state (that is caused by the input differential signal 708). The second amplifier 706 may avoid losing incoming input signals during such a recovery time.
A switch device 712 (“Switch” in
The switch device 712 may alternate the direction of flow of the signals (e.g., the output differential signal 710) from the first path 716 to the second path 718. Control of the switch device 712 may be provided by the control unit 112 (shown in
The output differential signals 710 received by the switch device 712 may be communicated to a comparison device 720 in the second path 718. Alternatively, the switch device 712 (or another component) may convert the differential signals 710 into a single-ended signal that is input into the comparison device 720. The comparison device 720 also receives the receive pattern signal 728 from the pattern generator 228 (shown in
The comparison device 720 compares the signals received from the switch device 712 with the receive pattern signal 728 to identify differences between the echo signal 226 and the receive pattern signal 728.
In one embodiment, the receive pattern signal 728 includes a pattern that is delayed by the time delay (e.g., the time of flight) identified by the coarse stage determination. The comparison device 720 may then compare this time-delayed pattern in the pattern signal 728 to the echo signal 226 (e.g., as modified by the amplifiers 704, 710) to identify overlaps or mismatches between the time-delayed pattern signal 728 and the echo signal 226.
In one embodiment, the comparison device 720 may include or represent a limiting amplifier that acts as a relatively high-speed XOR gate. An “XOR gate” includes a device that receives two signals and produces a first output signal (e.g., a “high” signal) when the two signals are different and a second output signal (e.g., a “low” signal) or no signal when the two signals are not different.
In another embodiment, the system may only include the coarse baseband processing circuits 716 or the fine baseband processing circuits 718. In this case, the switch 712 may also be eliminated. For example, this may be to reduce the cost or complexity of the overall system. As another example, the system may not need the fine accuracy and the rapid response of the coarse section 716 is desired. The coarse, fine and ultrafine stages may be used in any combination at different times in order to balance various performance metrics. Intelligent control can be manually provided by an operator or automatically generated by a processor or controller (such as the control unit 112) autonomously controlling the assembly 102 based on one or more sets of instructions (such as software modules or programs) stored on a tangible computer readable storage medium (such as a computer memory). The intelligent control can manually or automatically switch between which stages are used and/or when based on feedback from one or more other stages. For example, based on the determination from the coarse stage (e.g., an estimated time of flight or separation distance), the sensing assembly 102 may manually or automatically switch to the fine and/or ultrafine stage to further refine the time of flight or separation distance and/or to monitor movement of the target object 104.
With continued reference to
In one embodiment, the comparison device 720 generates the output signal 806 based on differences between the portion 800 of the echo signal 226 and the portion 802 of the time-delayed pattern signal 728. For example, when a magnitude or amplitude of both portions 800, 802 is “high” (e.g., has a positive value) or when the magnitude or amplitude of both portions 800, 802 is “low” (e.g., has a zero or negative value), the comparison device 720 may generate the output signal 806 to have a first value. In the illustrated example, this first value is zero. When a magnitude or amplitude of both portions 800, 802 differ (e.g., one has a high value and the other has a zero or low value), the comparison device 720 may generate the output signal 806 with a second value, such as a high value.
In the example of
The output signals 806 generated by the comparison device 720 represent temporal misalignment between the baseband echo signal 226 and the pattern signal 728 that is delayed by the time of flight or time delay measured by the coarse stage determination. The temporal misalignment may be an additional portion (e.g., to be added to) the time of flight of the transmitted signals 106 (shown in
The temporal misalignment between the baseband signal 226 and the pattern signal 728 may be referred to as a time lag. The time lag can be represented by the time periods 808, 810, 904, 906. For example, the time lag of the data stream 226 in
In order to measure the temporal misalignment between the baseband signal 226 and the time-delayed pattern signal, the output signals 806 may be communicated from the conversion device 720 to one or more filters 722. In one embodiment, the filters 722 are low-pass filters. The filters 722 generate energy signals 724 that are proportional to the energy of the output signals 806. The energy of the output signals 806 is represented by the size (e.g., width) of waveforms 812, 910 in the output signals 806. As the temporal misalignment between the baseband signal 226 and the pattern signal 728 increases, the size (and energy) of the waveforms 812, 910 increases. As a result, the amplitude and/or energy conveyed or communicated by the energy signals 724 increases. Conversely, as the temporal misalignment between the baseband signal 226 and the time-delayed pattern signal 728 decreases, the size and/or amplitude and/or energy of the waveforms 812, 910 also decreases. As a result, the energy conveyed or communicated by the energy signals 724 decreases.
As another example, the above system could be implemented using the opposite polarity, such as with an XNOR comparison device that produces “high” signals when the baseband signal 226 and the time-delayed pattern signal 728 are the same and “low” when they are different. In this example, as the temporal misalignment between the baseband signal 226 and the pattern signal 728 increases, the size (and energy) of the waveforms 812, 910 decreases. As a result, the amplitude and/or energy conveyed or communicated by the energy signals 724 decreases. Conversely, as the temporal misalignment between the baseband signal 226 and the time-delayed pattern signal 728 decreases, the size, amplitude, and/or energy of the waveforms 812, 910 also increases. As a result, the energy conveyed or communicated by the energy signals 724 increases.
The energy signals 724 may be communicated to measurement devices 726 (“ADC” in
The control unit 112 (or other component that receives the output signal 710) may examine the measured energy of the energy signals 724 and calculate the additional portion of the time of flight represented by the temporal misalignment between the baseband signal 226 and the time-delayed pattern signal 728. The control unit 112 also may calculate the additional portion of the separation distance 110 that is associated with the temporal misalignment. In one embodiment, the control unit 112 compares the measured energy to one or more energy thresholds. The different energy thresholds may be associated with different amounts of temporal misalignment. Based on the comparison, a temporal misalignment can be identified and added to the time of flight calculated using the coarse stage determination described above. The separation distance 110 may then be calculated based on the combination of the coarse stage determination of the time of flight and the additional portion of the time of flight from the fine stage determination.
The measurement devices 726 may digitize the energy signals 724 to produce the energy data output signals 702. When the output signals 702 are received from the measurement devices 726 (shown in
The different energy thresholds 1106 are associated with different temporal misalignments between the echo signal 226 and the time-delayed pattern signal 728 in one embodiment. For example, the energy threshold 1106A may represent a temporal misalignment of 100 picoseconds, the energy threshold 1106B may represent a temporal misalignment of 150 picoseconds, the energy threshold 1106C may represent a temporal misalignment of 200 picoseconds, the energy threshold 1106D may represent a temporal misalignment of 250 picoseconds, and so on. For example, 724B may be the result of the situation shown in
The measured energy of the output signal 702 can be compared to the thresholds 1106 to determine if the measured energy exceeds one or more of the thresholds 1106. The temporal misalignment associated with the largest threshold 1106 that is approached or reached or represented by the energy of the output signal 702 may be identified as the temporal misalignment between the echo signal 226 and the time-delayed pattern signal 728. In one embodiment, no temporal alignment may be identified for output signals 702 having or representing energies that are less than the threshold 1106A.
The energy thresholds 1106 may be established by positioning target objects 104 (shown in
In addition or as an alternate to performing the fine stage determination of the time of flight, the ultrafine stage may be used to refine (e.g., increase the resolution of) the time of flight measurement, track movement, and/or detect movement of the target object 104 (shown in
As described above, the ultrafine stage determination may alternatively or additionally involve a similar process as the coarse stage determination. For example, the coarse stage determination may examine the I channel of the receive pattern and the data stream to determine correlation values of different subsets of the data stream and, from those correlation values, determine a subset of interest and a corresponding time-of-flight, as described herein. The ultrafine stage determination can use the Q channel of the receive pattern and the data stream to determine correlation values of different subsets of the data stream and, from those correlation values, determine a subset of interest and a time-of-flight. The times-of-flight from the I channel and Q channel can be combined (e.g., averaged) to calculate a time of flight and/or separation distance to the target. The correlation values calculated by the ultrafine stage determination can be used to calculate an additional time delay that can be added to the time delays from the coarse stage and/or the fine stage to determine a time of flight and/or separation distance to the target. Alternatively or additionally, the correlation values of the waveforms in the I channel and Q channel can be examined to resolve phases of the echoes in order to calculate separation distance or motion of the target.
In operation, the echo signal 224 is received by the front end receiver 218 and is separated into separate I and Q signals 1206, 1208 (also referred to herein as I and Q channels). Each separate I and Q signal 1206, 1208 includes the corresponding I or Q component of the echo signal 224 and can be processed and analyzed similar to the signals described above in connection with the baseband processing system 232 shown in
Similar to as described above in connection with the switch device 712 (shown in
As described above, the energies of the signals output from the comparison devices 1218 can pass through the filters 1220 and be measured by the measurement devices 1222 to determine each of the temporal misalignments associated with the I and Q components of the echo signal 226 and the receive pattern signal. These temporal misalignments can be added together and added to the time of flight determined by the coarse stage determination. The sum of the temporal misalignments and the time of flight from the coarse stage determination can be used by the baseband processor 232 to calculate the separation distance 110 (shown in
In one embodiment, the ultrafine stage determination described above can be used to determine relatively small movements that change the separation distance 110 (shown in
where φ denotes the phase and I is the I projection 1320 and Q is the Q projection 1321. The carrier phase or the change in carrier phase can be used to calculate the distance or change in distance through the equation:
where λ is the wavelength of the carrier frequency and φ is the phase expressed in degrees as calculated from Equation 2 above.
The baseband processor 232 (shown in
The coarse, fine, and/or ultrafine stage determinations described above may be used in a variety of combinations. For example, the coarse stage determination may be used to calculate the separation distance 110 (shown in
As another example, if the separation distance 110 (shown in
Returning to the discussion of the system 100 shown in
A first digitized echo signal 1400 in
A correlation window 1406 includes a sequence 1414 of bits that can be compared to either digitized echo signal 1400, 1402 to determine a subset of interest, such as the subsets of interest 1408, 1410, in order to determine times of flight to the respective target objects 104 (shown in
In one embodiment, a mask 1412 can be applied to the sequence 1414 of bits in the correlation window 1406 to modify the sequence 1414 of bits in the correlation window 1406. The mask 1412 can eliminate or otherwise change the value of one or more of the bits in the correlation window 1406. The mask 1412 can include a sequence 1416 of bits that are applied to the correlation window 1406 (e.g., by multiplying the values of the bits) to create a modified correlation window 1418 having a sequence 1420 of bits that differs from the sequence 1414 of bits in the correlation window 1406. In the illustrated example, the mask 1412 includes a first portion of the first three bits (“101”) and a second portion of the last three bits (“000”). Alternatively, another mask 1412 may be used that has a different sequence of bits and/or a different length of the sequence of bits. Applying the mask 1412 to the correlation window 1406 eliminates the last three bits (“011”) in the sequence 1414 of bits in the correlation window 1406. As a result, the sequence 1420 of bits in the modified correlation window 1418 includes only the first three bits (“101”) of the correlation window 1418. In another embodiment, the mask 1412 adds additional bits to the correlation window 1406 and/or changes values of the bits in the correlation window 1406.
The sequence 1420 of bits in the modified correlation window 1418 can be used to change the sequence of bits in the pattern signal 230 (shown in
The modified correlation window 1418 can then be compared with the additional digitized echo signal 1422 to identify subsets of interest associated with the different target objects 104 (shown in
In operation, when transmitted signals 106 reflect off multiple target objects 104, the pattern transmitted in the signals 106 can be modified relatively quickly between successive bursts of the transmitted signals 106 when one or more of the target objects 104 cannot be identified from examination of the digitized echo signal 226. The modified pattern can then be used to distinguish between the target objects 104 in the digitized echo signal 740 using the correlation window that includes the modified pattern.
In another embodiment, the digital pulse sequence of bits included in a transmitted signal 106 (shown in
Several series-fed arrays 1506 are conductively coupled in parallel to form the array 1502 in the illustrated embodiment. The numbers of unit cells 1504 and series-fed arrays 1506 shown in
The front end 200 of the sensing assembly 102 may be housed in an enclosure 1602, such as a metal or otherwise conductive housing, with radio transmissive windows 1604 over the antennas 1500. Alternatively, the front end 200 may be housed in a non-metallic (e.g., dielectric) enclosure. The windows over the antennas 1500 may not be cut out of the enclosure 1602, but may instead represent portions of the enclosure 1602 that allows the transmitted signals 106 and echoes 108 pass through the windows 1604 from or to the antennas 1500.
The enclosure 1602 may wrap around the antennas 1500 so that the antennas are effectively recessed into the conducting body of the enclosure 1602, which can further improve isolation between the antennas 1500. Alternatively, in the case of a non-conducting enclosure 1602, the antennas 1500 may be completely enclosed by the enclosure 1602 and extra metal foil, and/or absorptive materials, or other measures may be added to improve isolation between the antennas 1500. In one embodiment, if the isolation is sufficiently high, the transmit and receiving antennas 1500 can be operated at the same time if the returning echoes 108 are sufficiently strong. This may be the case when the target is at very close range, and can allow for the sensing assembly 102 to operate without a transmit/receive switch.
The antenna 1500 may be positioned on a surface of a substrate 1706 that supports the antenna 1500. A conductive ground plane 1708 may be disposed on an opposite surface of the substrate 1706, or in another location.
The cover layer 1700 may be separated from the antenna 1500 by an air gap 1704 (“Air” in
This lensing effect can permit transmitted signals 106 and/or echoes 108 to pass through additional layers 1702 of materials (e.g., insulators such as Teflon, polycarbonate, or other polymers) that are positioned between the antenna 1500 and the target object 104 (shown in
In one embodiment, the substrate 1708 may have a thickness dimension between the opposite surfaces that is thinner than a wavelength of the carrier signal of the transmitted signals 106 and/or echoes 108. For example, the thickness of the substrate 1708 may be on the order of 1/20th of a wavelength. The thicknesses of the air gap 1704 and/or superstrate 1700 may be larger, such as ⅓ of the wavelength. Either one or both of the air gap 1704 and the superstrate 1700 may also be removed altogether.
One or more embodiments of the system 100 and/or sensing assembly 102 described herein may be used for a variety of applications that use the separation distance 110 and/or time of flight that is measured by the sensing assembly 102. Several specific examples of applications of the system 100 and/or sensing assembly 102 are described herein, but not all applications or uses of the system 100 or sensing assembly 102 are limited to those set forth herein. For example, many applications that use the detection of the separation distance 110 (e.g., as a depth measurement) can use or incorporate the system 100 and/or sensing assembly 102.
As another example, the sensing assembly 102 may communicate transmitted signals 106 that penetrate into the body of the patient 2300 and sense the motion or absolute position of various internal structures, such as the heart. Many of these positions or motions can be relatively small and subtle, and the sensing assembly 102 can use the ultrafine stage determination of motion or the separation distance 110 to sense the motion or absolute position of the internal structures.
Using the non-contact sensing assembly 102 also may be useful for situations where it is impossible or inconvenient to use wired sensors on the patient 2300 (e.g., sensors mounted directly to the test subject, connected by wires back to a medical monitor). For example, in high-activity situations where conventional wired sensors may get in the way, the sensing assembly 102 may monitor the separation distance 110 and/or motion of the patient 2300 from afar.
In another example, the sensing assembly 102 can be used for posture recognition and overall motion or activity sensing. This can be used for long-term observation of the patient 2300 for the diagnosis of chronic conditions, such as depression, fatigue, and overall health of at-risk individuals such as the elderly, among others. In the case of diseases with relatively slow onset, such as depression, the long term observation by the sensing assembly 102 may be used for early detection of the diseases. Also, since the unit can detect the medical parameters or quantities without anything being mounted on the patient 2300, the sensing assembly 102 may be used to make measurements of the patient 2300 without the knowledge or cooperation of the patient 2300. This could be useful in many situations, such as when dealing with children who would be made upset if sensors are attached to them. It may also give an indication of the mental state of a patient 2300, such as their breath becoming rapid and shallow when they become nervous. This would give rise to a remote lie-detector functionality.
In another embodiment, data generated by the sensing assembly 102 may be combined with data generated or obtained by one or more other sensors. For example, calculation of the separation distance 110 by the sensing assembly 102 may be used as a depth measurement that is combined with other sensor data. Such combination of data from different sensors is referred to herein as sensor fusion, and includes the fusing of two or more separate streams of sensor data to form a more complete picture of the phenomena or object or environment that is being sensed.
As one example, separation distances 110 calculated using the sensing assembly 102 may be combined with two-dimensional image data acquired by a camera. For example, without the separation distances 110, a computer or other machine may not be able to determine the actual physical size of the objects in a two-dimensional image.
For example, the sensing systems 2500 can acquire or measure information (e.g., light levels, radiation, moisture, heat, and the like) from the target objects 104A, 104B and the separation distances 110A, 1106 to the target objects 104A, 104B. The separation distances 110A, 1106 can be used to correct or calibrate the measured information. For example, if the target objects 104A, 104B both provide the same light level, radiation, moisture, heat, and the like, the different separation distances 110A, 1106 may result in the sensing systems 2500A, 2500B measuring different light levels, radiation, moisture, heat, and the like. With the sensing assembly 102 (shown in
As another example, the sensing system 2500 may include a reflective pulse oximetry sensor and the sensing assembly 102. Two or more different wavelengths of light are directed at the surface of the target object 104 by the system 2500 and a photo detector of the system 2500 examines the scattered light. The ratio of the reflected power can be used to determine the oxygenation level of the blood in the target object 104. Instead of being directly mounted (e.g., engaged to) the body of the patient that is the target object 104, the sensing system 2500 may be spaced apart from the body of the patient.
The surface of the patient body can be illuminated with light sources and the sensing assembly 102 (shown in
In another embodiment, the sensing assembly 102 and/or system 100 shown in
The examples of sensor fusion described herein are not limited to just the combination of the sensing assembly 102 and one other sensor. Additional sensors may be used to aggregate the separation distances 110 and/or motion detected by the sensing assembly 102 with the data streams acquired by two or more additional sensors. For example, audio data (from a microphone), video data (from a camera), and the separation distances 110 and/or motion from the sensing assembly 102 can be aggregated to give a more complete understanding of a physical environment.
At 2702, a determination is made as to whether to use to the coarse stage determination of the time of flight and/or separation distance. For example, an operator of the system 100 (shown in
At 2704, an oscillating signal is mixed with a coarse transmit pattern to create a transmitted signal. For example, the oscillating signal 216 (shown in
At 2706, the transmitted signal is transmitted toward a target object. For example, the transmitting antenna 204 (shown in
At 2708, echoes of the transmitted signal that are reflected off the target object are received. For example, the echoes 108 (shown in
At 2710, the received echoes are down converted to obtain a baseband signal. For example, the echoes 108 (shown in
At 2712, the baseband signal is digitized to obtain the coarse receive data stream. For example, it may pass through the baseband processor 232 including the digitizer 730 to produce the digitized echo signal 740.
At 2714, a correlation window (e.g., a coarse correlation window) and a coarse mask are compared to the data stream to identify a subset of interest. Alternatively, the mask (e.g., a mask to eliminate or change one or more portions of the data stream) may not be used. In one embodiment, the coarse correlation window 320 (shown in
At 2716, a time of flight of the transmitted signal and echo is calculated based on a time delay of the subset of interest. This time of flight can be referred to as a coarse time of flight. As described above, the subset of interest can be associated with a time lag (td) between transmission of the transmitted signal 106 (shown in
At 2718, a determination is made as to whether the fine stage determination of the separation distance is to be used. For example, a determination may be made automatically or manually to use the fine stage determination to further refine the measurement of the separation distance 110 (shown in
At 2720, an oscillating signal is mixed with a digital pulse sequence to create a transmitted signal. As described above, the transmit pattern that is used in the fine stage may be different from the transmit pattern used in the coarse stage. Alternatively, the transmit pattern may be the same for the coarse stage and the fine stage.
At 2722, the transmitted signal is communicated toward the target object, similar to as described above in connection with 2706.
At 2724, echoes of the transmitted signal that are reflected off the target object are received, similar to as described above in connection with 2708.
At 2726, the received echoes are down converted to obtain a baseband signal. For example, the echoes 108 (shown in
At 2728, the baseband signal 226 is compared to a fine receive pattern. The fine receive pattern may be delayed by the coarse time of flight, as described above. For example, instead of comparing the baseband signal with the receive pattern with both the baseband signal and the receive pattern having the same starting or initial time reference, the receive pattern may be delayed by the same time as the time delay measured by the coarse stage determination. This delayed receive pattern also may be referred to as a “coarse delayed fine extraction pattern” 728.
At 2730, a time lag between the fine data stream and the time delayed receive pattern is calculated. This time lag may represent the temporal overlap or mismatch between the waveforms in the fine data stream and the time delayed receive pattern, as described above in connection with
At 2732, the time of flight measured by the coarse stage (e.g., the “time of flight estimate”) is refined by the time lag. For example, the time lag calculated at 2730 can be added to the time of flight calculated at 2716. Alternatively, the time lag may be added to a designated time of flight, such as a time of flight associated with or calculated from a designated or known separation distance 110 (shown in
At 2734, the time of flight (that includes the time lag calculated at 2732) is used to calculate the separation distance from the target object, as described above. Flow of the method 2700 may then return to 2702 in a loop-wise manner. The above methods can be repeated for the I and Q channels separately or in parallel using parallel paths as in
In one embodiment, performance of the fine stage determination (e.g., as described in connection with 2720 through 2732) is performed on one of the I or Q components of channels of the transmit signal and the echo signal, as described above. For example, the I channel of the echo signal 226 (shown in
As described above, the ultrafine stage determination may alternatively or additionally involve a similar process as the coarse stage determination. For example, the coarse stage determination may examine the I channel of the receive pattern and the data stream to determine correlation values of different subsets of the data stream and, from those correlation values, determine a subset of interest and a corresponding time-of-flight, as described herein. The ultrafine stage determination can use the Q channel of the receive pattern and the data stream to determine correlation values of different subsets of the data stream and, from those correlation values, determine a subset of interest and a time-of-flight, as described above. The times-of-flight from the I channel and Q channel can be combined (e.g., averaged) to calculate a time of flight and/or separation distance to the target. The correlation values calculated by the ultrafine stage determination can be used to calculate an additional time delay that can be added to the time delays from the coarse stage and/or the fine stage to determine a time of flight and/or separation distance to the target. Alternatively or additionally, the correlation values of the waveforms in the I channel and Q channel can be examined to resolve phases of the echoes in order to calculate separation distance or motion of the target.
It is to be understood that the above description is intended to be illustrative, and not restrictive. For example, the above-described embodiments (and/or aspects thereof) may be used in combination with each other. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the inventive subject matter without departing from its scope. While the dimensions and types of materials described herein are intended to define the parameters of the inventive subject matter, they are by no means limiting and are exemplary embodiments. Many other embodiments will be apparent to one of ordinary skill in the art upon reviewing the above description. The scope of the inventive subject matter should, therefore, be determined with reference to the appended clauses, along with the full scope of equivalents to which such clauses are entitled. In the appended clauses, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.” Moreover, in the following clauses, the terms “first,” “second,” and “third,” etc. are used merely as labels, and are not intended to impose numerical requirements on their objects. Further, the limitations of the following clauses are not written in means-plus-function format and are not intended to be interpreted based on 35 U.S.C. § 112, sixth paragraph, unless and until such clause limitations expressly use the phrase “means for” followed by a statement of function void of further structure.
This written description uses examples to disclose several embodiments of the inventive subject matter and also to enable one of ordinary skill in the art to practice the embodiments of inventive subject matter, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the inventive subject matter is defined by the clauses, and may include other examples that occur to one of ordinary skill in the art. Such other examples are intended to be within the scope of the clauses if they have structural elements that do not differ from the literal language of the clauses, or if they include equivalent structural elements with insubstantial differences from the literal languages of the clauses.
The foregoing description of certain embodiments of the present inventive subject matter will be better understood when read in conjunction with the appended drawings. To the extent that the figures illustrate diagrams of the functional blocks of various embodiments, the functional blocks are not necessarily indicative of the division between hardware circuitry. Thus, for example, one or more of the functional blocks (for example, processors or memories) may be implemented in a single piece of hardware (for example, a general purpose signal processor, microcontroller, random access memory, hard disk, and the like). Similarly, the programs may be stand-alone programs, may be incorporated as subroutines in an operating system, may be functions in an installed software package, and the like. The various embodiments are not limited to the arrangements and instrumentality shown in the drawings.
As used herein, an element or step recited in the singular and proceeded with the word “a” or “an” should be understood as not excluding plural of said elements or steps, unless such exclusion is explicitly stated. Furthermore, references to “one embodiment” of the present inventive subject matter are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features. Moreover, unless explicitly stated to the contrary, embodiments “comprising,” “including,” or “having” an element or a plurality of elements having a particular property may include additional such elements not having that property.
This patent application is a continuation of U.S. patent application Ser. No. 16/224,922 filed 19 Dec. 2018; which is a continuation of U.S. patent application Ser. No. 14/428,075 filed 13 Mar. 2015, now U.S. Pat. No. 10,206,610 issued 19 Feb. 2019; which is a national stage of International Application PCT/US2013/063253 filed 3 Oct. 2013; which claims a benefit of priority to U.S. Provisional Patent Application 61/710,397 filed 5 Oct. 2012; each of which are hereby incorporated by reference in their entirety for all purposes.
Number | Name | Date | Kind |
---|---|---|---|
3019430 | Norman | Jan 1962 | A |
3447155 | Anderson | May 1969 | A |
3778160 | Wolcott | Dec 1973 | A |
3796208 | Bloice | Mar 1974 | A |
3906213 | Meriaux et al. | Sep 1975 | A |
4513748 | Nowogrodzki et al. | Apr 1985 | A |
4699508 | Bolkow et al. | Oct 1987 | A |
4829317 | Shinkawa | May 1989 | A |
5118134 | Mattes et al. | Jun 1992 | A |
5131754 | Hasegawa | Jul 1992 | A |
5175553 | Le Garrec | Dec 1992 | A |
5274271 | McEwan | Dec 1993 | A |
5321409 | Walker | Jun 1994 | A |
5345471 | McEwan | Sep 1994 | A |
5361070 | McEwan | Nov 1994 | A |
5366241 | Kithil | Nov 1994 | A |
5457394 | McEwan | Oct 1995 | A |
5465094 | McEwan | Nov 1995 | A |
5471198 | Newham | Nov 1995 | A |
5510800 | McEwan | Apr 1996 | A |
5512834 | McEwan | Apr 1996 | A |
5517198 | McEwan | May 1996 | A |
5519400 | McEwan | May 1996 | A |
5523760 | McEwan | Jun 1996 | A |
5573012 | McEwan | Nov 1996 | A |
5589838 | McEwan | Dec 1996 | A |
5661490 | McEwan | Aug 1997 | A |
5754144 | McEwan | May 1998 | A |
5757320 | McEwan | May 1998 | A |
5766208 | McEwan | Jun 1998 | A |
5774091 | McEwan | Jun 1998 | A |
5793327 | Carnes et al. | Aug 1998 | A |
5805110 | McEwan | Sep 1998 | A |
5966090 | McEwan | Oct 1999 | A |
6031421 | McEwan | Feb 2000 | A |
6031504 | McEwan | Feb 2000 | A |
6060915 | McEwan | May 2000 | A |
6072427 | McEwan | Jun 2000 | A |
6137438 | McEwan | Oct 2000 | A |
6191724 | McEwan | Feb 2001 | B1 |
6195047 | Richards | Feb 2001 | B1 |
6218982 | Shirai et al. | Apr 2001 | B1 |
6219596 | Fukae et al. | Apr 2001 | B1 |
6373428 | McEwan | Apr 2002 | B1 |
6414627 | McEwan | Jul 2002 | B1 |
6426716 | McEwan | Jul 2002 | B1 |
6456231 | McEwan | Sep 2002 | B1 |
6462705 | McEwan | Oct 2002 | B1 |
6466125 | Richards et al. | Oct 2002 | B1 |
6492933 | McEwan | Dec 2002 | B1 |
6504483 | Richards et al. | Jan 2003 | B1 |
6531977 | McEwan | Mar 2003 | B2 |
6535161 | McEwan | Mar 2003 | B1 |
6580385 | Winner et al. | Jun 2003 | B1 |
6587072 | Gresham et al. | Jul 2003 | B1 |
6661342 | Hall et al. | Dec 2003 | B2 |
6747599 | McEwan | Jun 2004 | B2 |
6788206 | Edwards | Sep 2004 | B1 |
6879281 | Gresham et al. | Apr 2005 | B2 |
6900732 | Richards | May 2005 | B2 |
6914552 | McEwan | Jul 2005 | B1 |
7145384 | Point et al. | Dec 2006 | B2 |
7151478 | Adams et al. | Dec 2006 | B1 |
7199747 | Jenkins et al. | Apr 2007 | B2 |
7224944 | McEwan | May 2007 | B2 |
7275173 | Lindt | Sep 2007 | B2 |
7446695 | McEwan | Nov 2008 | B2 |
7463181 | Wintermantel | Dec 2008 | B2 |
7529290 | Hiromori | May 2009 | B2 |
7567200 | Osterweil | Jul 2009 | B1 |
7679562 | Shirakawa | Mar 2010 | B2 |
7864105 | Matsumoto | Jan 2011 | B2 |
7887089 | Breed et al. | Feb 2011 | B2 |
7916066 | Osterweil | Mar 2011 | B1 |
7990310 | Kwak et al. | Aug 2011 | B2 |
7994968 | McEwan | Aug 2011 | B2 |
8068051 | Osterweil | Nov 2011 | B1 |
8115673 | McEwan | Feb 2012 | B1 |
8134497 | Janky et al. | Mar 2012 | B2 |
8466827 | Nanami | Jun 2013 | B2 |
8599018 | Kellen et al. | Dec 2013 | B2 |
8884813 | Bangera et al. | Nov 2014 | B2 |
20010035836 | Miceli et al. | Nov 2001 | A1 |
20010035837 | Fullerton et al. | Nov 2001 | A1 |
20020176511 | Fullerton et al. | Nov 2002 | A1 |
20020189336 | McEwan | Dec 2002 | A1 |
20030025626 | McEwan | Feb 2003 | A1 |
20030045816 | Foxlin | Mar 2003 | A1 |
20030112174 | Kim | Jun 2003 | A1 |
20030117310 | Kikuchi et al. | Jun 2003 | A1 |
20030151542 | Steinlechner et al. | Aug 2003 | A1 |
20030193430 | Gresham et al. | Oct 2003 | A1 |
20030210181 | Hager et al. | Nov 2003 | A1 |
20030220570 | Ragnarsdottir | Nov 2003 | A1 |
20040102919 | Eckel et al. | May 2004 | A1 |
20040168512 | McEwan | Sep 2004 | A1 |
20040227661 | Godsy | Nov 2004 | A1 |
20040249258 | Tupin, Jr. et al. | Dec 2004 | A1 |
20050013465 | Southall et al. | Jan 2005 | A1 |
20050046584 | Breed | Mar 2005 | A1 |
20050078029 | Okamura et al. | Apr 2005 | A1 |
20050179586 | Klinnert et al. | Aug 2005 | A1 |
20050258999 | Jenkins et al. | Nov 2005 | A1 |
20050275583 | Mikami et al. | Dec 2005 | A1 |
20060012432 | Point et al. | Jan 2006 | A1 |
20060055584 | Waite et al. | Mar 2006 | A1 |
20060058683 | Chance | Mar 2006 | A1 |
20060287117 | Dilz et al. | Dec 2006 | A1 |
20070008213 | Adams et al. | Jan 2007 | A1 |
20070177704 | McEwan | Aug 2007 | A1 |
20070192391 | McEwan | Aug 2007 | A1 |
20070210955 | McEwan | Sep 2007 | A1 |
20080048905 | McEwan | Feb 2008 | A1 |
20080089557 | Iwaki et al. | Apr 2008 | A1 |
20080272957 | Schoeberl | Nov 2008 | A1 |
20080294019 | Tran | Nov 2008 | A1 |
20080316085 | Rofougaran et al. | Dec 2008 | A1 |
20080316090 | Shirakawa | Dec 2008 | A1 |
20090003412 | Negoro et al. | Jan 2009 | A1 |
20090017839 | Kim | Jan 2009 | A1 |
20090045952 | Bahari | Feb 2009 | A1 |
20090074031 | Fukuda | Mar 2009 | A1 |
20090079617 | Shirakawa et al. | Mar 2009 | A1 |
20090079618 | Harwood et al. | Mar 2009 | A1 |
20090180518 | Ishii et al. | Jul 2009 | A1 |
20090212995 | Wu et al. | Aug 2009 | A1 |
20100007548 | Kunzler et al. | Jan 2010 | A1 |
20100048313 | Mooney | Feb 2010 | A1 |
20100063676 | Ito | Mar 2010 | A1 |
20100073221 | McEwan | Mar 2010 | A1 |
20100130873 | Yuen et al. | May 2010 | A1 |
20100141506 | Gulden et al. | Jun 2010 | A1 |
20100152600 | Droitcour | Jun 2010 | A1 |
20100214157 | McEwan | Aug 2010 | A1 |
20100214158 | McEwan | Aug 2010 | A1 |
20100241009 | Petkie | Sep 2010 | A1 |
20100274100 | Behar | Oct 2010 | A1 |
20110221624 | Kavaler | Sep 2011 | A1 |
20120105268 | Smits et al. | May 2012 | A1 |
20120179011 | Moon et al. | Jul 2012 | A1 |
20120212366 | Alalusi | Aug 2012 | A1 |
20130335259 | Yasugi et al. | Dec 2013 | A1 |
20140058255 | Mase et al. | Feb 2014 | A1 |
20140104315 | Kapler et al. | Apr 2014 | A1 |
20150002507 | Ambrus et al. | Jan 2015 | A1 |
20160054566 | Osterhout et al. | Feb 2016 | A1 |
20160077202 | Hirvonen et al. | Mar 2016 | A1 |
20160249316 | Kudekar et al. | Aug 2016 | A1 |
20160299234 | Zeng et al. | Oct 2016 | A1 |
20160363648 | Mindell et al. | Dec 2016 | A1 |
20170097413 | Gillian et al. | Apr 2017 | A1 |
20170238136 | Smith | Aug 2017 | A1 |
20170307726 | Mazzaro et al. | Oct 2017 | A1 |
20170310758 | Davis | Oct 2017 | A1 |
20170336210 | Rahman et al. | Nov 2017 | A1 |
20170350973 | Dunik et al. | Dec 2017 | A1 |
20180233145 | Bathiche et al. | Aug 2018 | A1 |
Number | Date | Country |
---|---|---|
0717290 | Jun 1996 | EP |
1865337 | Dec 2007 | EP |
2068475 | Jun 2009 | EP |
2178320 | Apr 2010 | EP |
2460691 | Jun 2008 | GB |
2489299 | Sep 2012 | GB |
04193158 | Jul 1992 | JP |
05256936 | Oct 1993 | JP |
2006027229 | Feb 1994 | JP |
07035849 | Feb 1995 | JP |
2007333325 | Dec 1995 | JP |
10197624 | Jul 1998 | JP |
2010197624 | Jul 1998 | JP |
2000121726 | Apr 2000 | JP |
2003043137 | Feb 2003 | JP |
2003290154 | Oct 2003 | JP |
2005270570 | Oct 2005 | JP |
2005351862 | Dec 2005 | JP |
2007005849 | Jan 2007 | JP |
2007189436 | Jul 2007 | JP |
2008513145 | May 2008 | JP |
JPWO2006106774 | Sep 2008 | JP |
2008256568 | Oct 2008 | JP |
2009098097 | May 2009 | JP |
2009128288 | Jun 2009 | JP |
2009192516 | Aug 2009 | JP |
2009528859 | Aug 2009 | JP |
2009265031 | Nov 2009 | JP |
2009538720 | Nov 2009 | JP |
2009290725 | Dec 2009 | JP |
2010508128 | Mar 2010 | JP |
2010230643 | Oct 2010 | JP |
2011007518 | Jan 2011 | JP |
2011519657 | Jul 2011 | JP |
2011234830 | Nov 2011 | JP |
2015-535782 | Dec 2015 | JP |
6273662 | Feb 2018 | JP |
WO2002097468 | Dec 2002 | WO |
WO2004047630 | Jun 2004 | WO |
WO2006034211 | Mar 2006 | WO |
WO2007101343 | Sep 2007 | WO |
WO2007143535 | Dec 2007 | WO |
WO2008057883 | May 2008 | WO |
WO2008102291 | Aug 2008 | WO |
WO2009136341 | Nov 2009 | WO |
WO2008038790 | Jan 2010 | WO |
WO2009019892 | Oct 2010 | WO |
WO2011143631 | Nov 2011 | WO |
WO2012054086 | Apr 2012 | WO |
WO2012115220 | Aug 2012 | WO |
WO2014048668 | Apr 2014 | WO |
Entry |
---|
Davies, Google ATAP's Ivan Poupyrev Talks Projects Soli and Jacquard, downloaded from internet https://www.tomshardware.com/news/google-atap-project-soli-jacquard.29371.html, published Jun. 25, 2019 (10 page). |
Europe Search Report dated Jul. 4, 2014 for corresponding Application No. 12782290.6 filed Feb. 20, 2012 (7 pages). |
Europe Search Report dated Apr. 26, 2016 for corresponding Application No. 13844396.5 filed Oct. 3, 2013 (7 pages). |
Europe Search Report dated Nov. 29, 2016 for corresponding Application No. 12782290.6 filed Feb. 20, 2012 (7 pages). |
Europe Search Report dated Jan. 31, 2018 for corresponding Application No. 13844396.5 filed Oct. 3, 2013 (7 pages). |
Europe Search Report dated Feb. 2, 2018 for corresponding Application No. 17199843.8 filed Feb. 20, 2012 (12 pages). |
Europe Search Report dated May 16, 2018 for corresponding Application No. 17199843.8 filed Feb. 20, 2012 (11 pages). |
Europe Search Report dated Oct. 1, 2019 for corresponding Application No. 17199843.8 filed Feb. 20, 2012 (5 pages). |
Europe Search Report dated Mar. 31, 2021 for corresponding Application No. 17199843.8 filed Feb. 20, 2012 (6 pages). |
Europe Search Report dated May 6, 2021 for corresponding Application No. 20196709.8 filed 2013-110-13 (9 pages). |
European Search Report dated Jun. 12, 2022, for corresponding Application No. 17199843.8 filed Feb. 20, 2012 (6 pages). |
Final Office Action dated Oct. 25, 2022 for corresponding U.S. Appl. No. 15/798,911, filed Oct. 31, 2017 (8 pages). |
International Preliminary Report on Patentability dated Apr. 12, 2022 for PCT/US2020/054598 filed Oct. 7, 2020 (10 pages). |
International Preliminary Report on Patentability dated Apr. 27, 2021 for PCT/US2019/057544 filed Oct. 23, 2019 (10 pages). |
International Search Report and Written Opinion dated Dec. 12, 2012 for Application No. PCT/US2012/025800 filed Feb. 20, 2012 (7 pages). |
International Search Report and Written Opinion dated Jan. 20, 2014 for Application No. PCT/US2013/063253 filed Oct. 3, 2013 (7 pages). |
International Search Report and Written Opinion dated Dec. 16, 2019 for Application No. PCT/US2019/051676 filed Sep. 18, 2019 (15 pages). |
International Search Report and Written Opinion dated Jan. 21, 2020 for Application No. PCT/US2019/057544 filed Oct. 23, 2019 (16 pages). |
International Search Report and Written Opinion dated Nov. 3, 2020 for Application No. PCT/US2020/044217 filed Jul. 30, 2019 (16 pages). |
International Search Report and Written Opinion dated Dec. 8, 2020 for Application No. PCT/US2020/054598 filed Oct. 7, 2020 (16 pages). |
Japan Decision of Refusal Mar. 26, 2021 for corresponding JP2019-153214 filed Aug. 23, 2019 (2 pages). |
Japan Office Action dated Dec. 24, 2015 for corresponding Application No. 2013-554668 filed Feb. 20, 2012 (4 pages). |
Japan Office Action dated Aug. 28, 2017 for corresponding Application No. 2015-535782 filed Oct. 3, 2013 (3 pages). |
Japan Office Action dated Aug. 30, 2017 for corresponding Application No. 2016-192041 filed Sep. 29, 2016 (6 pages). |
Japan Office Action dated May 7, 2019 for corresponding Application No. 2018-102805 filed Sep. 29, 2016 (6 pages). |
Japan Office Action dated Apr. 1, 2020 for corresponding Application No. 2018-102805 filed Sep. 29, 2016 (6 pages). |
Japan Office Action dated Aug. 28, 2020 for corresponding Application No. 2019-153214 filed Aug. 23, 2019 (2 pages). |
Japan Office Action dated Oct. 26, 2021 for corresponding Application No. 2021-034678 filed Aug. 23, 2019 (2 pages). |
Japan Office Action dated Nov. 22, 2021 for corresponding Application No. 2020-200335 filed Oct. 3, 2013 (3 pages). |
Japan Office Action dated Apr. 13, 2022 for corresponding Application No. 2021-034678 filed Mar. 4, 2021 (12 pages). |
Japan Office Action dated Aug. 1, 2022 for corresponding Application No. 2020-200335 filed Dec. 2, 2020 (6 pages). |
Japan Office Action dated Dec. 5, 2022 for corresponding Application No. 2021-034678 filed Mar. 4, 2021 (10 pages). |
Japan Office Action dated Nov. 22, 2021 for corresponding Application No. 2020-200335 filed Dec. 2, 2020 (6 pages). |
United States Final Office Action dated Oct. 21, 2020 for corresponding U.S. Appl. No. 16/702,871, filed Dec. 4, 2019 (18 pages). |
United States Final Office Action dated Apr. 8, 2021 for corresponding U.S. Appl. No. 15/798,911, filed Oct. 31, 2017 (9 pages). |
United States Final Office Action dated Feb. 22, 2022 for corresponding U.S. Appl. No. 16/702,871, filed Dec. 4, 2019 (15 pages). |
United States Final Office Action dated Mar. 21, 2022 for corresponding U.S. Appl. No. 16/262,117, filed Jan. 30, 2019 (9 pages). |
United States Non Final Office Action dated Mar. 21, 2018 for corresponding U.S. Appl. No. 14/428,075, filed Mar. 13, 2015 (17 pages). |
United States Nonfinal Office Action dated Oct. 18, 2018 for corresponding U.S. Appl. No. 16/103,423, filed Aug. 14, 2018 (13 pages). |
United States Nonfinal Office Action dated Feb. 10, 2020 for corresponding U.S. Appl. No. 16/702,871, filed Dec. 4, 2019 (19 pages). |
United States Nonfinal Office Action dated Aug. 3, 2020 for corresponding U.S. Appl. No. 15/798,911, filed Oct. 31, 2017 (11 pages). |
United States Nonfinal Office Action dated Jun. 11, 2021 for corresponding U.S. Appl. No. 16/702,871, filed Dec. 4, 2019 (20 pages). |
United States Nonfinal Office Action dated Sep. 7, 2021 for corresponding U.S. Appl. No. 16/262,117, filed Jan. 30, 2019 (24 pages). |
United States Nonfinal Office Action dated Nov. 8, 2021 for corresponding U.S. Appl. No. 16/028,091, filed Jul. 5, 2018 (25 pages). |
United States Nonfinal Office Action dated Nov. 10, 2021 for corresponding U.S. Appl. No. 16/661,068, filed Oct. 23, 2019 (24 pages). |
United States Nonfinal Office Action dated Dec. 13, 2021 for corresponding U.S. Appl. No. 15/798,911, filed Oct. 31, 2017 (12 pages). |
United States Notice of Allowance dated Jan. 6, 2015 for corresponding U.S. Appl. No. 13/400,261, filed Feb. 20, 2012 (8 pages). |
United States Notice of Allowance dated Oct. 22, 2018 for corresponding U.S. Appl. No. 14/428,075, filed Mar. 13, 2015 (9 pages). |
Number | Date | Country | |
---|---|---|---|
20230397842 A1 | Dec 2023 | US |
Number | Date | Country | |
---|---|---|---|
61710397 | Oct 2012 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 16224922 | Dec 2018 | US |
Child | 18231211 | US | |
Parent | 14428075 | US | |
Child | 16224922 | US |