Patent Application
20250040818
This application claims the benefit of European Patent Application No. 23188773, filed on Jul. 31, 2023, which application is hereby incorporated herein by reference.
The present disclosure relates to radar data processing. In particular, examples relate to an apparatus, a radar system, a method, and a program for estimating a heart rate.
Vital signs, such as a breathing rate and heart rate, are physical measurements of the body that can be used to monitor a general health status of a living being. While breathing rates and heart rates may be measured by various methods, such methods under certain circumstances might not be feasible. In particular, for a mostly static target, a high velocity resolution for detection of a breathing rate may be more easily obtained compared to a heart rate, which is more challenging. Also for radar systems, the detection of the heart rate faces many challenges, due to environmental noise, harmonic components of the breathing rate, and random body movement. Thus, there is a demand for improved radar data processing for heart rate detection.
This demand is satisfied by the subject matter of the independent claims. Further beneficial embodiments are given by the dependent claims.
According to a first aspect, the present disclosure relates to an apparatus for measuring a heart rate of a target in a field of view of a radar sensor. the apparatus comprises processing circuitry configured to determine a frequency spectrum corresponding to time-domain data based on a receive signal. The processing circuitry is further configured to determine a breathing rate frequency based on a maximum amplitude of the frequency spectrum. Furthermore, the processing circuitry is configured to determine a plurality of section-frequency-spectra, each corresponding to a respective section of the time-domain data, based on the breathing rate frequency, and to determine an average of the section-frequency-spectra to obtain an averaged-frequency-spectrum. The processing circuitry is further configured to subtract the averaged-frequency-spectrum from the frequency spectrum to obtain a difference spectrum, and to estimate a heart rate of the target based on a maximum amplitude of the difference spectrum.
According to a second aspect, the present disclosure relates to a radar system comprising an apparatus as described herein and a radar sensor configured to generate the receive signal.
According to a third aspect, the present disclosure relates to a method for measuring a heart rate of a target in a field of view of a radar sensor. The method comprises determining a frequency spectrum corresponding to time-domain data based on a receive signal. The method further comprises determining a breathing rate frequency based on a maximum amplitude of the frequency spectrum. Furthermore, the method comprises determining a plurality of section-frequency-spectra, each corresponding to a respective section of the time-domain data, based on the breathing rate frequency, and further comprises determining an average of the section-frequency-spectra to obtain an averaged-frequency-spectrum. Furthermore, the method comprises subtracting the averaged-frequency-spectrum from the frequency spectrum to obtain a difference spectrum and estimating a heart rate of the target based on a maximum amplitude of the difference spectrum.
According to a fourth aspect, the present disclosure relates to a program having a program code for performing a method as described herein when the program is executed on a processor or programmable hardware.
Some examples of apparatuses and/or methods will be described in the following by way of example only, and with reference to the accompanying figures, in which
Some examples are now described in more detail with reference to the enclosed figures. However, other possible examples are not limited to the features of these embodiments described in detail. Other examples may include modifications of the features as well as equivalents and alternatives to the features. Furthermore, the terminology used herein to describe certain examples should not be restrictive of further possible examples.
Throughout the description of the figures same or similar reference numerals refer to same or similar elements and/or features, which may be identical or implemented in a modified form while providing the same or a similar function. The thickness of lines, layers and/or areas in the figures may also be exaggerated for clarification.
When two elements A and B are combined using an “or”, this is to be understood as disclosing all possible combinations, i.e. only A, only B as well as A and B, unless expressly defined otherwise in the individual case. As an alternative wording for the same combinations, “at least one of A and B” or “A and/or B” may be used. This applies equivalently to combinations of more than two elements.
If a singular form, such as “a”, “an” and “the” is used and the use of only a single element is not defined as mandatory either explicitly or implicitly, further examples may also use several elements to implement the same function. If a function is described below as implemented using multiple elements, further examples may implement the same function using a single element or a single processing entity. It is further understood that the terms “include”, “including”, “comprise” and/or “comprising”, when used, describe the presence of the specified features, integers, steps, operations, processes, elements, components and/or a group thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, processes, elements, components and/or a group thereof.
The apparatus 100 comprises processing circuitry 110 and, optionally, interface circuitry 120. In case interface circuitry 120 is present, the interface circuitry 120 may be communicatively coupled (e.g., via a wired or wireless connection) to the processing circuitry 110, e.g., for data exchange between the interface circuitry 120 and the processing circuitry 110. The interface circuitry 120 may be any device or means for communicating or exchanging data. In case the apparatus 100 comprises the interface circuitry 120, the interface circuitry 120 may be configured to receive data indicating a receive signal of the radar sensor. For instance, the interface circuitry 120 may be communicatively coupled to the radar sensor or to a storage device storing the data. The interface circuitry 120 may receive the data, e.g., via a wired or wireless coupling to the radar sensor or the storage device.
Depending on the specific implementation, the apparatus 100 may dispense with the interface circuitry 120. For example, the processing circuitry 110 may determine said data. For instance, the processing circuitry 110 may be integrated into the radar sensor. The radar sensor may be any device that uses radio waves to, e.g., detect and locate objects. The radar sensor may be an, e.g., FMCW (frequency modulated continuous wave) radar sensor. For instance, the radar sensor may be configured to emit, by a transmitter, a radio frequency signal (Tx signal) into a field of view (a scene) of the radar sensor and receive, by a receiver, a reflection (echo; Rx signal) of the radar frequency signal. The radar sensor or an external device coupled to the radar sensor may generate radar data based on the received reflection of the radio frequency signal by, e.g., sampling the received reflection by means of an analog-to-digital converter (ADC).
The radio frequency signal (i.e. receive signal) may be in the form of a plurality of chirps, including parts of a received reflection signal which are correlated to respective emitted chirps. A chirp may be a radio frequency signal that varies in frequency over time. The frequency of the chirp may be swept over a specific frequency range, e.g., over the chirp bandwidth. For instance, the chirp may be a linearly modulated signal, i.e., a signal of which the frequency increases or decreases linearly over time.
The receive signal may be any type of signal that is generated in a receiver of the radar sensor. The receive signal may be an intermediate frequency, IF, signal, which may e.g., be created by mixing the reflection with a local oscillator signal at a specific frequency. The processing circuitry 110 may determine data indicating the receive signal by, e.g., sampling the receive signal and perform further processing of the data within the radar sensor. The processing circuitry 110 may optionally modify the sampled receive signal in a pre-processing step, e.g., for noise-reduction, DC-removal (direct current) or alike. For instance, the apparatus 100 may comprise memory configured to store such generated data.
The radar sensor may mix the received echo with a replica of the emitted signal using a mixer to produce an IF signal xIF (t) (e.g., a beat signal). The radar sensor may comprise one or more antennas to receive the reflected signal and the radar sensor may comprise an amplifier to receive the reflected signals from its antennas. The beat signal xIF (t) may be filtered with a low-pass filter (LPF) and then sampled by the ADC. The ADC may advantageously be capable of sampling the filtered beat signals xout(t) with a sampling frequency that is smaller than the frequency of the received signal received by the receiving antennas.
Alternatively, the processing circuitry 110 may partially determine the data. For instance, the processing circuitry 110 may determine a first part of the data, whereas at least one external processing circuitry may determine at least a second part of the data. The processing circuitry 110 and the external processing circuitry may, e.g., be connected within a distributed computing environment for jointly determining the data. In this case, the processing circuitry 110 may either be integrated into the radar sensor or may be external to the radar sensor. The processing circuitry 110 may receive the second part of the data, e.g., via an interface to the external processing circuitry such as interface circuitry 120, and further process the first and the second part of the data.
In another alternative, the processing circuitry 110 is partially integrated into the radar sensor and is partially external to the radar sensor. In such cases, the interface circuitry 120 is optional. The processing circuitry 110 may, for instance, comprise a first part (first processing circuitry) which is integrated into the radar sensor and a second part (second processing circuitry) which is external to the radar sensor. In this case, the determination of the data and/or further processing may be performed by the first and second part of the processing circuitry 110 in a distributed manner.
The processing circuitry 110 may be, e.g., a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which or all of which may be shared, a digital signal processor (DSP) hardware, an application specific integrated circuit (ASIC), a microcontroller or a field programmable gate array (FPGA). The processing circuitry 110 may optionally be coupled to, e.g., read only memory (ROM) for storing software, random access memory (RAM) and/or non-volatile memory.
As part of the above-described radar system and its multiple components, the various embodiments of the apparatus 100 described herein are provided for determining a heart rate and a breathing rate of a target in the field of view of the radar sensor. For this, the apparatus 100 is configured to receive data based on the receive signal, including time-domain data 130, and perform processing steps based the time-domain data 130. The time-domain data 130 may be checked and/or processed into other forms of data before being used for generating a plurality of frequency spectra that may be analyzed for determining the heart rate and breathing rate of the target.
The processing circuitry 110 of the apparatus 100 is configured to determine a frequency spectrum corresponding to the time-domain data 130 and to determine a breathing rate frequency based on a maximum amplitude therein. The breathing rate frequency may be determined by an analysis of the frequency spectrum, including a search for and identification of a maximum amplitude of the frequency spectrum. The breathing rate frequency may be determined by identifying a frequency bin, within which the amplitude is at a maximum value for the entire frequency spectrum. The breathing rate frequency may be set to a value within a frequency range corresponding to the frequency bin, which may be equivalent to a center frequency within the frequency bin.
The frequency spectrum may take the form of a variety of frequency spectrum types. For example, the frequency spectrum may be an amplitude spectrum, wherein the frequency spectrum comprises a plurality of amplitudes, each amplitude corresponding to a respective frequency bin (or frequency component). In a particular example, the frequency spectrum may be a magnitude spectrum representing a magnitude or absolute value of complex amplitudes in a frequency domain. As such, the magnitude spectrum may provide information about an intensity or strength corresponding to a frequency component without considering phase information. Alternatively, the frequency spectrum may be a power spectrum representing a power of each frequency component. The frequency spectrum may also be a normalized spectrum, representing the amplitudes or powers of the frequency components scaled to a specific reference or normalized to a specific value. The various types of frequency spectra presented for the above-described frequency spectrum may also analogously apply to all other frequency spectra used in further processing.
The processing circuitry 110 is further configured to determine a plurality of section-frequency-spectra, each corresponding to a respective section of the time-domain data 130. Furthermore, the plurality of section-frequency spectra are determined based on the breathing rate frequency. For example, the time-domain data 130 may be divided into a plurality of sections, wherein each section corresponds to a time interval, which may be the same amount of time for each section. The time interval may be defined by the inverse of the breathing rate frequency. For example, the inverse of the breathing rate frequency may provide a time interval equivalent to an amount of time between successive breaths taken at a frequency equivalent to the breathing rate frequency. As such, each section of the time-domain data 130 may include a single breath at a similar time instance within the time interval.
The processing circuitry 110 is further configured to determine an average of the section-frequency-spectra to obtain an averaged-frequency-spectrum. Each of the section-frequency-spectra may comprise a common frequency range and a common number of frequency bins (e.g. a same resolution). The processing circuitry 110 may be configured to sum the respective amplitudes of each section-frequency-spectrum and divide the sum by the number of section-frequency-spectra. More specifically, the respective amplitude values of each section-frequency-spectrum that each correspond to a particular frequency bin, such as a frequency bin of an equivalent position or index, may be summed and divided by the number of section-frequency-spectra. This may also be done for each frequency bin of the frequency spectra, so that each frequency bin has associated therewith an averaged amplitude value. The plurality of averaged amplitude values, each associated with its respective frequency bin, may be used for determining the averaged-frequency-spectrum.
The processing circuitry 110 is configured to subtract the averaged-frequency-spectrum from the (original) frequency spectrum to obtain a difference spectrum. For the subtraction, the respective amplitude values of each frequency bin of the averaged-frequency-spectrum may be subtracted from the respective amplitude values of each frequency bin of the (original) frequency spectrum. For example, an amplitude value of a particular frequency bin of the averaged-frequency-spectrum may be subtracted from an amplitude value of an analogous frequency bin with an equivalent position or index of the (original) frequency spectrum. This may be done for each frequency bin of the averaged-frequency-spectrum and the (original) frequency spectrum to obtain a difference value for each frequency bin. The respective difference values may then be used to obtain a full difference spectrum. The difference spectrum may also be considered as a difference dataset with respective difference-in-amplitude values, wherein the difference dataset may be plotted according to the respective frequency bins to provide a difference (frequency) spectrum.
Furthermore, the processing circuitry 110 is configured to estimate a heart rate of the target based on a maximum amplitude of the difference spectrum. The heart rate may correspond to a specific heart rate frequency, which may correspond to the maximum amplitude of the difference spectrum. Similar to finding the breathing rate frequency based on a maximum amplitude, the heart rate frequency may be determined by an analysis of the difference spectrum, including a search for and identification of a maximum amplitude of the difference spectrum. The heart rate frequency may be determined by identifying a frequency bin, within which the amplitude is at a maximum value for the entire difference spectrum. The heart rate frequency may be determined as a value within a frequency range corresponding to the frequency bin, which may be equivalent to a center frequency within the frequency bin.
As such, the processing circuitry 110 is configured to obtain a heart rate frequency (e.g., estimate a heart rate) and obtain (i.e. determine) a breathing rate frequency. The heart rate frequency and breathing rate frequencies may each be used to provide health information related to the target in the field of view of the radar sensor. Such heart rate and breathing rate frequencies may also be recorded multiple times, which may performed over a period of days, weeks, months, or years to obtain long-term health information related to the target. The apparatus 100 may comprise other features that may improve the accuracy of the obtained breathing rate and/or heart rate frequencies. Such features will be discussed in greater depth with reference to
The frequency spectrum 200-1 is shown with a first amplitude peak 210-1 corresponding to the maximum amplitude. The maximum amplitude peak 210-1 is located at a first frequency, which may correspond to the breathing rate frequency. For example, the breathing rate frequency may be 0.75 Hz, which would correspond to 45 breaths per 60 seconds, or 45 breaths per minute, which is a breathing rate within a usual range for a newborn baby (usually ranging from 30 to 60 breaths per minute). The maximum amplitude is depicted to correspond to a value of 45 on the x-axis, or 45 breaths per minute.
The frequency spectrum 200-1 is also shown with a second amplitude peak 210-2 at a second frequency that is twice the frequency of the breathing rate frequency. It can be seen that the second frequency corresponds to 90 breaths per minute (1.5 Hz). As such, the second amplitude peak 210-2 may also be referred to as the first harmonic of the breathing rate frequency. Furthermore, the frequency spectrum 200-1 is shown with a third amplitude peak 210-3 at a third frequency that is three times the frequency of the breathing rate frequency. It can be seen that the third frequency corresponds to 135 breaths per minute (2.25 Hz). As such, the third amplitude peak 210-3 may also be referred to as the second harmonic of the breathing rate frequency. The use of the term “amplitude peak” refers to a local maximum that is significantly greater than other possible local maxima, so as to distinguish itself from other local maxima that are not significantly greater in amplitude than various local minima of the same spectrum.
Additionally, the frequency spectrum 200-1 comprises a fourth peak 212 that corresponds to a fourth frequency that is slightly below the third introduced frequency. The fourth peak may correspond to a heart rate. Within the depicted scale, the fourth peak may correspond to a heart rate that is approximately 125 beats per minute (approx. 2.08 Hz), slightly below the second harmonic frequency of 135 breaths per minute (2.25 Hz). The heart rate of 125 beats per minute is a heart rate within a usual range for a resting heart rate of a newborn baby (usually ranging from 100 to 160 beats per minute).
An average of the plurality of the section-frequency-spectra may be calculated as previously described to obtain the averaged-frequency-spectrum 200-2, which is depicted in
The major difference between the depicted (original) frequency spectrum 200-1 and the averaged-frequency-spectrum 200-2 is the lack of a heart rate component. Rather the particular frequency component in the averaged-frequency-spectrum 200-2 near the heart rate frequency of 125 beats per minute (approx. 2.08 Hz) only comprises noise. In the averaged-frequency-spectrum 200-2, this frequency is depicted with an arrow and label showing that the “heart rate component has been filtered”. The heart rate may be effectively filtered or at least mostly filtered when choosing a time interval for each of the section-frequency-spectra that corresponds to the breathing rate of the target, i.e. the inverse of the breathing rate frequency.
In this scenario, each section of the time-domain data 130 will have amplitude values corresponding to a breathing in and breathing out of the target at an analogous position of the respective section. As such, the breathing rate frequency will be depicted very similarly in the averaged-frequency spectrum 200-2 compared to the (original) frequency spectrum 200-1. Furthermore, the harmonic components appear mostly unaffected. When dividing the time-domain data 130 into sections corresponding to the inverse of the fundamental frequency, each section captures a complete period of the signal. In other words, by dividing the time-domain data 130 according to the fundamental frequency, each section aligns with a specific phase of the fundamental signal. As a result, the influence by the harmonic components, being integer multiples of the fundamental frequency, tend to remain consistent within each section of the time-domain data 130, assuming the harmonics are stable.
However, the heart rate component, unlike the first and second harmonics, is located at a position in the (original) frequency spectrum 200-1, such that its corresponding data in the plurality of sections of the time-domain data 130 does not remain aligned across the sections. Depending on its position and strength, it may be fully averaged out to no longer be visible in the averaged-frequency-spectrum 200-2 or may be mostly averaged out to be only partially visible in the averaged-frequency-spectrum 200-2. In either case, the difference spectrum may comprise a maximum amplitude that corresponds to a heart rate estimate.
While the previously described techniques including the determination of the difference spectrum and the heart rate estimation based on the maximum amplitude thereof may provide an adequately accurate heart rate estimation, this estimation may be improved by further processing techniques of the time-domain data 130. Such techniques will be discussed in greater depth with reference to
The pre-specified number of frames may correspond to a pre-specified amount of time (i.e. a time window) that is designated for receiving the receive signal. More specifically, the time-domain data 130 may correspond to the time window. For the example of an FMCW radar sensor, if the time window is 10 seconds, the time-domain data 130 may correspond to 10 seconds of the radar system emitting frequency-modulated continuous wave signals and receiving the corresponding echoed (i.e. reflected) signals. For example, in such a case, the apparatus 100 may receive 500 frames in 10 seconds, or 1 frame every 0.02 seconds. The time window of 10 seconds may be chosen for a desired balance. On one hand, it may be beneficial to obtain more time-domain data for more accurate averaging. On the other hand, the time window should not be extended too long, where a random body movement of the target or other possible disturbances have a higher likelihood to cause the time-domain data 130 to become invalid. For example, in the case of the target being a baby, it becomes increasingly unlikely after 10 seconds that the baby will remain adequately still for obtaining accurate measurements for valid time-domain data 130. The time window may be adjusted according to each use case or a user input, such as ranging between 8 and 12 seconds.
A second check may be a noise check 320-1. The noise check 320-1 may be used to determine whether the time-domain data 130 based on the receive signal is below a pre-specified noise-level threshold 320-2. For example, the processing circuitry 110 may be configured to apply a high-pass filter for a processing of the time-domain data 130. Based on the high-pass filter processing, the processing circuitry 110 may be configured to calculate an average amplitude of the high-pass filtered time-domain data 130 to obtain a noise-level average. The noise-level average may then be checked if it is below the noise-level threshold 320-2. Proceeding forward, the processing circuitry 110 may be configured to determine the frequency spectrum 200-1 if the noise-level average is under a pre-specified noise-level threshold 320-2.
A third check may be a range check 330-1. The range check 330-1 may be used to determine whether the receive signal comprises sufficient time-domain data from a pre-specified valid range 330-2. If it is determined that the time-domain data 130 or a required amount of the time-domain data 130 does not correspond to the pre-specified range, the processing circuitry 110 may be configured to record new measurements for new time-domain data 130 (i.e. waiting for the next frame 350 for new measurements). The range of the time-domain data 130 may be determined by a range discrete Fourier transformation (DFT) of the time-domain data 130. The range-DFT may lead to the apparatus 100 obtaining a range buffer divided into a plurality of range bins. More specifically, the results of the range-DFT may be stored in a memory of the apparatus 100, wherein portions of the time-domain data 130 are stored according to the respective range bins. A first range bin may correspond to a first range from the radar sensor, a second range bin may correspond to a second range from the radar sensor, etc.
As part of or in addition to the range check 330-1, the range-DFT may also be performed on the basis of a plurality of frames of the time-domain data 130, which will be discussed in greater detail with reference to
For example, the processing circuitry 110 may be configured to perform mean removal of the time-domain data 130. More specifically, the processing circuitry 110 may be configured to calculate a mean value of amplitude values of the time-domain data 130 and to subtract the mean value from the amplitude values to generate mean-removed time-domain data. This may shift the previous values, which may include an offset, to be centered on zero. Generally, using a form of mean-removed time-domain data 130 may enable generating the frequency spectra 200-1; 200-2 more accurately. If applicable, the mean-removed version of the time-domain data 130 may also be used to perform further checks more accurately, such as confirming a valid breathing rate and heart rate for the target (provided in
The mean removal may be applied for the entire time-domain data 130 or on a per frame basis. Using a per frame basis, the mean value of amplitude values may be a mean value of the given frame, which may be subtracted from the amplitude values of the given frame. This may generate mean-removed time domain data specifically for the given frame. As depicted, a first portion of mean-removed time-domain data 412-1 may correspond to a first time frame 410-1, a second portion of mean-removed time-domain data 412-2 may correspond to a second time frame 410-2, up to an n-th portion 412-n for an n-th frame 410-n. Furthermore, the previously described range-DFT may be performed using the various portions of mean-removed time-domain data. Further references to the processing of the time-domain data 130 may apply either to mean-removed or non-mean-removed time domain data 130.
The processing circuitry 110 may be configured to perform a separate range-DFT for each frame 410-1; 410-2; 410-n (i.e. on a per frame basis) of the time-domain data 130 . . . . If mean-removal of the time-domain data is applied, the range-DFT may be performed for each portion of the mean-removed time-domain data 412-1; 412-2; 412-n corresponding to their respective frames. As such, the range-DFT may be referred to as a short time range-DFT being applied to short time range data (respective frames or portions) of the time-domain data 130. Each of the respective range-DFTs being performed on the respective frames or portions of the (mean-removed) time-domain data 130 may lead to generating a respective range buffer 420-1; 420-2; 420-n, as depicted, corresponding to each frame in
The processing circuitry 110 may identify a particular range bin for each range buffer with the greatest value (e.g., greatest amplitude value or mean-removed amplitude value) of the (mean-removed) time-domain data 130. More specifically, a collection of corresponding range bins across the plurality of (e.g., 500) range buffers may be chosen, wherein the collection of range bins each correspond to an equivalent index or position within its respective range buffer. The processing circuitry 110 may be configured to choose such a collection of range bins corresponding to a common position or index, each (or most) of the range bins comprising a maximum amplitude of the (mean-removed) time-domain data 130. As such, the collection of corresponding (i.e. analogously positioned) range bins may also be referred to simply as a collective “range bin” or “identified range bin”, referring specifically to the common position within each range buffer.
The processing circuitry 110 may be configured to check whether or not the identified range bin 422 is within a pre-specified selection of range bins for the respective range buffers. The pre-specified selection of range bins may correspond to a range of measurement within an acceptable range of distance away from the radar sensor. Such an acceptable range of distance may be, for example, between 1 to 1.5 meters, or another acceptable range of a similar scale (e.g., between 0.5 meters and 3 meters). If the identified range bin 422 is not within the pre-specified selection of range bins, the time-domain data 130 may be considered to be invalid and a new iteration of measurements by the radar system may begin (i.e. wait for the next frame 350). On the other hand, if the identified range bin 422 is indeed within the pre-specified selection of range bins, the time-domain data 130 may be considered to be within a pre-specified valid range 330-2.
Based on a determination of a valid range of the time-domain data 130, the processing circuitry 110 may determine the frequency spectrum 200-1 and the averaged-frequency-spectrum 200-2 for the (mean-removed) time-domain data 130 corresponding to the identified range bin 422 for each range buffer 420-1; 420-32; 420-n. In other words, the processing circuitry 110 may be configured to transform the (mean-removed) time-domain data 130 by a range-DFT, select a portion of the data generated, and determine the frequency spectrum 200-1 based on the selected data, which corresponds to a certain portion of the time-domain data 130.
Since the range-DFT may be applied across the range buffers, it may be considered as being applied to long time-domain data. By focusing the Doppler-DFT to the identified range bin 422, the frequency spectra 200-1; 200-2 may be more accurately determined.
The processing steps presented in
The breathing pattern checks 340-1a; 340-1b, which may be applied to the time-domain data 130 or specifically, the mean-removed time-domain data, as previously described, corresponding to the identified range bin 422. Alternatively, the processing circuitry 110 may be configured to perform mean removal after the range-DFT, for which a mean value of amplitude values in the identified range bin 422 is calculated and subtracted from the mean value from the amplitude values in the identified range bin 422. This may shift the information of the time-domain data 130 corresponding to the identified range bin 422 from the previous values to zero-centered values to generate the mean-removed range-time dataset 430 for the identified range bin 422. The mean-removed range-time dataset 430 may enable faster processing for further checks of the data to ensure that a valid breathing rate and heart rate may be found for the target in the field of view of the radar sensor. If such checks are passed, then the frequency spectrum may be determined specifically based on the mean-removed range-time dataset 430.
The exemplary range-time profile 500 based on the mean-removed range-time dataset 430 (with mean-removal performed before or after the range-DFT) shows visually how this shifts the time-domain data 130 to be centered around zero. The x-axis of the range-time profile 500 may be correspond to any arbitrary unit of time (e.g., seconds), while the y-axis of the range-time profile 500 may correspond to a complex number amplitude directly proportional to a range (i.e. distance or displacement), which may be measured using an appropriate unit of thereof.
In fact, the range-time profile 500 may provide a visual depiction of a breathing pattern when plotted. A breathing of the target may include a breathing in and a breathing out, which may repeat in a steady pattern over time. For example, while breathing in, air is inhaled and a chest of the target may expand, which may thus decrease the range (i.e. distance or displacement) between the target and the radar sensor. On the other hand, while breathing out, air is exhaled and the chest of the target may contract, which may thus increase the range (distance or displacement) between the target and the radar sensor. Since a breathing pattern must follow a recognizable pattern, certain checks may be implemented to ensure a proper breathing of the target. With a consistent breathing, the processing steps outlined in
The mean-removed range-time dataset 430 may correspond to the mean-removed range-time profile 500, which may depict a variation in the range (distance or displacement) that may be checked for consistency. In the first breathing check 340-1a presented in
For example, a first peak-to-peak value corresponding to an increase in range (distance or displacement) starting at a time t=0 may be approximately 4.0×10−3 arbitrary units. A second peak-to-peak value corresponding to a decrease in range (distance or displacement) may be approximately 3.5×10−3 units. A third, fourth, and fifth peak-to-peak values may also approximately 3.5×10−3 units, while a sixth peak-to-peak value, for the third decrease in range (distance or displacement), may be approximately 4×10−3 units. A plurality of peak-to-peak values may be checked. Optionally, each peak-to-peak value may be checked, wherein a peak-to-peak is determined to correspond to any increase from a local minimum to a local maximum or to correspond to any decrease from a local maximum to a local minimum (wherein local minima or local maxima of a very narrow frequency range may be neglected). A peak-to-peak-threshold 340-2a (depicted as part of the breathing check verification in
In a second breathing check 340-1b in
Each inhalation may have associated therewith a derivative value. For example, a derivative value may correspond to change in units of range (distance or displacement) of the dataset from the local minimum to the local maximum divided by the change in time in seconds. The same may analogously be true for a change from a local maximum to a local minimum (wherein local minima or local maxima of a very narrow frequency range may be neglected, as previously described). Alternatively or additionally, a derivative value may correspond to a change in range (displacement) divided by a corresponding change in time, wherein the time interval within the range-time profile 500 for the corresponding range and time changes may be arbitrarily chosen. The processing circuitry 110 may be configured to determine any number of derivative values within the range-time profile 500, which may be from a local minimum to a local maximum or vice versa, or which may correspond to a narrower time interval within such points.
The processing circuitry 110 may be configured to determine a variance of such derivative values as a derivative-threshold 340-2b (depicted as part of the breathing check verification in
Generally, the five above-described checks 310-1; 320-1; 330-1; 340-1a; 340-1b may be used in any combination to ensure by means of their respective thresholds 310-2; 320-2; 330-2; 340-2a; 340-2b that the time-domain data 130 is valid data, which may be considered reliable for determining a breathing rate and a heart rate of the target within the field of view of the radar sensor. Such checks may be part of a pre-processing of the apparatus 100. If all applied checks are passed, the processing by the apparatus 100 may continue to its core-processing steps, as outlined in
A first optional processing step shown in
Whereas FIR filters may conventionally or usually be used for filtering (i.e. filtering by data processing) of time-domain data before a Doppler-DFT, the processing circuitry 110 may be specifically configured to apply the FIR filter after the Doppler-DFT to data of the generated frequency spectrum 200-1 (i.e. the time-domain data 130 in processed form as part of the frequency spectrum 200-1) to prevent a disturbance in signal processing by a vibrating object. For example, for a wave signal emitted by a frequency-modulated continuous wave, FMCW, radar, a frequency component in a Doppler-DFT analysis may correspond to a speed or velocity of an object in the field of view of the radar sensor, as derived from the Doppler Effect, describing how the frequency of an emitted and reflected wave may change when reflected off an object with a relative motion from the radar sensor.
While an FIR filter, such as an FIR-high-pass filter, may filter data before a Doppler-DFT in the time-domain according to a high speed or a particular speed component within the signal, an FIR filter applied to filter the data after a Doppler-DFT in the frequency domain may filter a high change of speed, or a particular change-of-speed component within the data. As such, a disturbance in the data by a vibration, which may correspond to a particular change of speed that causes movement in the previously described positive and negative directions, may be filtered. In general, various versions of the FIR filter may be applied to filter the influence of various vibrating objects in the field of view of the radar sensor, particular for vibration ranges of common household objects, such as an air-conditioning unit.
A second optional processing step shown in
Furthermore, the processing circuitry 110 may be configured to detect a third frequency corresponding to another local maximum of amplitude values in the frequency spectrum, specifically within a second pre-specified frequency range from three times the first frequency (i.e. second harmonic). In the (original) frequency spectrum 200-1 of
Such validity tests may also be performed related to the averaged-frequency-spectrum 200-2 in
If this is not the case, the processing circuitry 110 may be configured to determine a new breathing rate frequency based on a neighboring local maximum of amplitude values in the (original) frequency spectrum 200-1. For example, a highly localized spike may be present in the determined (original) frequency spectrum 200-1, which may have caused the assignment of a somewhat inaccurate value for the breathing rate frequency. To prevent this, the processing circuitry 110 may be configured to accept a breathing rate frequency that corresponds to an amplitude within a certain pre-specified range. If the breathing rate frequency is determined to be inaccurate (for any reason), the processing circuitry 110 may be configured to repeat its previously performed calculations for a following or previous local maximum from the previously labeled breathing rate frequency (wherein local minima or local maxima of a very narrow frequency range may be neglected, as previously described).
Such a local maximum may also be searched for within a customized frequency range that may correspond to a breathing rate that corresponds to the physical characteristics of the target. For example, a target that is a human newborn infant may have a known range for breathing rate frequency, given certain assumptions (e.g., being at rest). A newborn infant may have such a search range vary between 20 and 70 breaths per minute, an older baby/toddler between 15 and 40 breaths per minute, and an adult between 8 and 25 breaths per minute. Other physical characteristics besides the age of the target may also affect the customized range. Such configurations may be adjustable. Adjustable configurations may be adjusted by a user input received through a user interface, among other possible means.
The processing circuitry 110 may be configured to repeat the same processing steps for the new breathing rate frequency, such as for a following or previous local maximum, to obtain a new averaged-frequency-spectrum based on the new breathing rate frequency. Furthermore, the previously described checks for the maximum amplitude value of the new averaged-frequency-spectrum may be verified to correspond to a frequency that is within its pre-specified frequency range from the new breathing rate frequency. If this is not the case, another local maximum of the (original) frequency spectrum 200-1 may be checked if it is within the customized frequency range based on the target's physical characteristics, with the previously described steps repeated. If no local maximum within the customized frequency range meet the required checks, then the processing circuitry 110 may be configured to label the time-domain data 130 as invalid and to cancel further processing thereof, as well as to begin the procedure for recording a new iteration of time-domain data.
While multiple examples related to calculation thresholds have been explained in depth, the processing circuitry 110 may be configured to apply a variety of thresholds for ensuring that an accurate value of the breathing rate frequency and heart rate frequency is determined/estimated. Such thresholds may be applied in various combinations and at various stages of processing. As depicted in
More specifically, the processing circuitry 110 may be configured to perform a first correlation operation for the (original) frequency spectrum 200-1 to obtain a correlated (original) frequency spectrum (in
For example, in
The first, second, and/or third Gaussian windows may also be determined by a different means, such as by a pre-specified frequency range from the respective maximum, which may be customized. While respective first, second, and third Gaussian windows of the averaged-frequency-spectrum 200-2 may also be used in its correlation operation, they are not depicted for the sake of clarity in
Once the first, second, and third Gaussian windows have been determined, they may be applied to generate a windowed (original) frequency spectrum 730-2, depicted in
Furthermore, the above-described techniques may be repeated for the previously described second correlation operation. The second correlation operation may be between the averaged-frequency spectrum 200-2 and a windowed version of the averaged-frequency-spectrum (not depicted, analogous to 730-2) that is analogous to the windowed version of the (original) frequency spectrum 200-1. In other words, the averaged-frequency spectrum may also have a first, second, and third Gaussian window applied (not depicted, analogous to 732-1a; 732-1b; 732-2a; 732-2b; 732-3a; 732-3b), as previously described, to form its windowed version. The correlation operation may then be performed, also as previously described, to obtain a respective correlation spectrum 730-3b.
Additionally, the processing circuitry 110 may be configured to apply a window to the difference spectrum 200-3, and to search for the maximum amplitude 230-1 within the window of the difference spectrum 200-3 for estimation of the heart rate frequency. The window of the difference spectrum 200-3 may specify a frequency range corresponding to a range of possible heart rates of the target, which may also be customized (e.g., from a user input) according to the target in the field of view of the radar sensor.
The processing circuitry 110 may also be configured to record multiple heart rate estimations within a single recording iteration or briefly separated recording iterations. For example, given a time window of 10 seconds or a similar value (e.g., 8 to 12 seconds) for a measurement iteration, approximately 5 to 7 estimations for the heart rate of the target may be performed within one minute of time. If the target is adequately still with minimal random body movement, then all estimations of the respective measurement iterations may be determined to be valid. Alternatively, if too much random body movement has occurred in a time window, thereby preventing a measurement of valid data for the heart rate estimation, the time-domain data corresponding to that particular time window may be discarded and further measurement iterations may continue. For example, the processing circuitry 110 may be configured to stop measuring and immediately begin a new measurement iteration (e.g. of 10 seconds) if a threshold related to random body movement has been surpassed, which may include the checks and corresponding thresholds described in
In particular, the processing circuitry 110 may be configured to perform multiple heart rate estimations, wherein each estimation is applied to an iteration of a recursive model for obtaining a refined heart rate estimate. For example, a first estimated value for the heart rate of the target may be based on a first heart rate estimate with a first range of error. Given a second heart rate estimate of the same target immediately following the first heart rate estimate, wherein both estimates are based on respective time-domain data 130 under the same circumstances, the second heart rate may increase confidence of the previous estimate. Furthermore, the recursive model may determine that the heart rate of the target is a value within a smaller range of error.
The recursive model may continue to be applied, wherein further, e.g., five or more, estimated values for the heart rate of the target may be given. Based on each successive estimation, the recursive model may determine an increased probability that the heart rate of the target is within a narrowed range. Generally, such a narrowed range may be part of the refined heart rate estimate, wherein the final estimate is considered to be more accurate and/or precise.
In addition to a refined heart rate estimate, the apparatus 100 may also to be used to increase confidence in a breathing rate estimate (i.e. breathing rate determination). Based on many possible checks applying the breathing rate frequency and its first and second harmonics, a higher confidence in the breathing rate may be obtained. Furthermore, other possible checks (e.g., checks related to the heart rate), which may not even be possible without an accurate value of the breathing rate, may also be done and verified, further increasing confidence in the breathing rate.
In general, the apparatus 100, used in accordance with one of the various above-described configurations may provide a particular benefit of long-term monitoring biological signs of the same target. In the particular case of a newborn baby, heart rates and breathing rates may be measured and recorded multiple times and performed over many different days (or longer time spans), which may provide caregivers with more reliable information related to biological activity and overall health of the newborn baby. Such a benefit is also possible for older children and adults. A further benefit may be to better understand patterns of such biological activity in a variety of target types (e.g., human babies or adults, or also animals) and how they may be affected, particularly if other events occurring within an associated time span are also recorded. Generally, the apparatus 100 provides a new means for examining biological activity and health by a new technique using radar systems.
Although the apparatus 910 and the radar sensor 920 are depicted as separate blocks in
More details and aspects of the radar system 900 are explained in connection with the proposed technique or one or more examples described above, e.g., with reference to
More details and aspects of the method 1000 are explained in connection with the proposed technique or one or more examples described above, e.g., with reference to
In the following, some examples of the proposed technique are presented:
An example (e.g., example 1) relates to an apparatus comprising processing circuitry configured to determine a frequency spectrum corresponding to time-domain data based on a receive signal, determine a breathing rate frequency based on a maximum amplitude of the frequency spectrum, determine a plurality of section-frequency-spectra, each corresponding to a respective section of the time-domain data, based on the breathing rate frequency, determine an average of the section-frequency-spectra to obtain an averaged-frequency-spectrum, subtract the averaged-frequency-spectrum from the frequency spectrum to obtain a difference spectrum, and estimate a heart rate of the target based on a maximum amplitude of the difference spectrum.
Another example (e.g., example 2) relates to a previous example (e.g., example 1) or to any other example, further comprising that the processing circuitry is configured to apply a high-pass filter for a processing of the time-domain data, calculate an average amplitude of the high-pass filtered time-domain data to obtain a noise-level average, and determine the frequency spectrum if the noise-level average is under a pre-specified noise-level threshold.
Another example (e.g., example 3) relates to a previous example (e.g., one of the examples 1 or 2) or to any other example, further comprising that the processing circuitry is configured to calculate a mean value of amplitude values of the time-domain data, subtract the mean value from the amplitude values of the time-domain data to generate mean-removed time-domain data, and determine the frequency spectrum for the mean-removed time-domain data.
Another example (e.g., example 4) relates to a previous example (e.g., one of the examples 1 to 3) or to any other example, further comprising that the processing circuitry is configured to perform one or more range Fourier Transformations of the time-domain data to obtain one or more range buffers, each divided into a plurality of range bins, for each range buffer, identify a range bin associated with a maximum amplitude to determine a range-time dataset, and determine the frequency spectrum for the range-time dataset corresponding to the identified range bin.
Another example (e.g., example 5) relates to a previous example (e.g., example 4) or to any other example, further comprising that the processing circuitry is configured to check whether or not a peak-to-peak value for the range-time dataset is within pre-specified limits based on an average of a plurality of previous peak-to-peak values in the range-time dataset, and determine the frequency spectrum for the range-time dataset if the peak-to-peak value is within its pre-specified limits.
Another example (e.g., example 6) relates to a previous example (e.g., one of the examples 4 or 5) or to any other example, further comprising that the processing circuitry is configured to determine a plurality of derivative values corresponding to a rate of change of a range at a respective time of the range-time dataset, determine a variance of the plurality of derivative values, check that each of the derivative values is within a threshold determined by the variance, and determine the frequency spectrum for the range-time dataset if each derivative value is within the threshold.
Another example (e.g., example 7) relates to a previous example (e.g., one of the examples 1 to 6) or to any other example, further comprising that the processing circuitry is configured to determine the frequency spectrum for the time-domain data and to subsequently apply a finite-impulse-response filter to the amplitude values of the frequency spectrum.
Another example (e.g., example 8) relates to a previous example (e.g., one of the examples 1 to 7) or to any other example, further comprising that the processing circuitry is configured to determine the breathing rate frequency by detection of a first frequency corresponding to a maximum amplitude value of the frequency spectrum and detection of a second frequency corresponding to a local maximum of amplitude values in the frequency spectrum within a pre-specified frequency range from twice the first frequency.
Another example (e.g., example 9) relates to a previous example (e.g., example 8) or to any other example, further comprising that the processing circuitry is configured to verify that a maximum amplitude value of the averaged-frequency-spectrum corresponds to a frequency that is within a pre-specified frequency range from the breathing rate frequency.
Another example (e.g., example 10) relates to a previous example (e.g., example 9) or to any other example, further comprising that for the case that the maximum amplitude value of the averaged-frequency-spectrum corresponds to a frequency that is not within the pre-specified frequency range from the breathing rate frequency, the processing circuitry is configured to determine a new breathing rate frequency based on a neighboring local maximum of amplitude values in the frequency spectrum, obtain a new averaged-frequency-spectrum based on the new breathing rate frequency, verify that the maximum amplitude value of the new averaged-frequency-spectrum corresponds to a frequency that is within the pre-specified frequency range from the new breathing rate frequency.
Another example (e.g., example 11) relates to a previous example (e.g., one of the examples 1 to 10) or to any other example, further comprising that the processing circuitry is configured to perform a first correlation operation for the frequency spectrum to determine a correlated frequency spectrum and a second correlation operation for the averaged frequency spectrum to determine a correlated averaged-frequency-spectrum, wherein each correlation operation is between an original version and a windowed version of the respective spectrum, wherein the respective windowed version is altered by applying a first Gaussian window centered on a first frequency corresponding to a maximum amplitude of the spectrum, applying a second Gaussian window centered on a second frequency that is two times the first frequency, applying a third Gaussian window centered on a third frequency that is three times the first frequency, and nullifying all amplitude values outside of the three Gaussian windows, and subtract the correlated averaged-frequency-spectrum from the correlated frequency spectrum to obtain the difference spectrum.
Another example (e.g., example 12) relates to a previous example (e.g., one of the examples 1 to 11) or to any other example, further comprising that the processing circuitry is configured to perform multiple heart rate estimations, wherein each estimation is applied to an iteration of a recursive model for obtaining a refined heart rate estimate.
An example (e.g., example 13) relates to a radar system, comprising an apparatus according to a previous example (e.g., any one of examples 1 to 12) or to any other example, and a radar sensor configured to generate the receive signal.
An example (e.g., example 14) relates to a method, comprising determining a frequency spectrum corresponding to time-domain data based on a receive signal, determining a breathing rate frequency based on a maximum amplitude of the frequency spectrum, determining a plurality of section-frequency-spectra, each corresponding to a respective section of the time-domain data, based on the breathing rate frequency, determining an average of the section-frequency-spectra to obtain an averaged-frequency-spectrum, subtracting the averaged-frequency-spectrum from the frequency spectrum to obtain a difference spectrum, estimating a heart rate of the target based on a maximum amplitude of the difference spectrum.
An example (e.g. example 15) relates to a program having a program code for performing a method according to a previous example (e.g. example 14) when the program is executed on a processor or programmable hardware.
The aspects and features described in relation to a particular one of the previous examples may also be combined with one or more of the further examples to replace an identical or similar feature of that further example or to additionally introduce the features into the further example.
Examples may further be or relate to a (computer) program including a program code to execute one or more of the above methods when the program is executed on a computer, processor or other programmable hardware component. Thus, steps, operations or processes of different ones of the methods described above may also be executed by programmed computers, processors or other programmable hardware components. Examples may also cover program storage devices, such as digital data storage media, which are machine-, processor- or computer-readable and encode and/or contain machine-executable, processor-executable or computer-executable programs and instructions. Program storage devices may include or be digital storage devices, magnetic storage media such as magnetic disks and magnetic tapes, hard disk drives, or optically readable digital data storage media, for example. Other examples may also include computers, processors, control units, (field) programmable logic arrays ((F) PLAs), (field) programmable gate arrays ((F) PGAs), graphics processor units (GPU), application-specific integrated circuits (ASICs), integrated circuits (ICs) or system-on-a-chip (SoCs) systems programmed to execute the steps of the methods described above.
It is further understood that the disclosure of several steps, processes, operations or functions disclosed in the description or claims shall not be construed to imply that these operations are necessarily dependent on the order described, unless explicitly stated in the individual case or necessary for technical reasons. Therefore, the previous description does not limit the execution of several steps or functions to a certain order. Furthermore, in further examples, a single step, function, process or operation may include and/or be broken up into several sub-steps, -functions, -processes or -operations.
If some aspects have been described in relation to a device or system, these aspects should also be understood as a description of the corresponding method. For example, a block, device or functional aspect of the device or system may correspond to a feature, such as a method step, of the corresponding method. Accordingly, aspects described in relation to a method shall also be understood as a description of a corresponding block, a corresponding element, a property or a functional feature of a corresponding device or a corresponding system.
The following claims are hereby incorporated in the detailed description, wherein each claim may stand on its own as a separate example. It should also be noted that although in the claims a dependent claim refers to a particular combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of any other dependent or independent claim. Such combinations are hereby explicitly proposed, unless it is stated in the individual case that a particular combination is not intended. Furthermore, features of a claim should also be included for any other independent claim, even if that claim is not directly defined as dependent on that other independent claim.
| Number | Date | Country | Kind |
|---|---|---|---|
| 23188773 | Jul 2023 | EP | regional |
