Patent Application
20150313502
The present invention is directed to systems and methods for determining an arterial pulse wave transit time for a subject.
The ability to capture physiological signals is highly desirable in the healthcare industry. One physiological signal of importance is the arterial pulse transit time (PTT). This is important for many reasons, one of which is that the PTT correlates well with blood pressure and thus can provide healthcare professionals with vital information relating to blood velocity in the vascular network, blood vessel dilation over time, and vessel blockage between two points or regions. Moreover, localized PTT can be used as an indirect marker for assessing pathologic conditions such as peripheral vascular disease.
Accordingly, what is needed in this art are increasingly sophisticated systems and methods for determining a subject's arterial pulse transit time.
The following U.S. patents, U.S. patent applications, and Publications are incorporated herein in their entirety by reference.
“Determining A Total Number Of People In An IR Image Obtained Via An IR Imaging System”, U.S. Pat. No. 8,520,074, by Wang et al.
What is disclosed is a system and method for determining arterial pulse transit time. In one embodiment, a video is received comprising a plurality of time-sequential image frames of a region of exposed skin of a subject where a videoplethysmographic (VPG) signal can be registered by at least one imaging channel of the video device used to capture that video. Also received is an electrocardiogram (ECG) signal obtained using a sensor placed on the subject's body where a ECG signal can be obtained. Batches of image frames are processed to obtain a continuous VPG signal for the subject. Thereafter, temporally overlapping VPG and ECG signals are analyzed to obtain the pulse transit time between a reference point on the VPG signal and a reference point on the ECG signal. The pulse transit time can be used to assess pathologic conditions such as peripheral vascular disease.
Features and advantages of the above-described method will become readily apparent from the following detailed description and accompanying drawings.
The foregoing and other features and advantages of the subject matter disclosed herein will be made apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:
What is disclosed is a system and method for determining arterial pulse wave transit time for a subject.
“Plethysmography” is the study of relative blood volume changes in blood vessels.
A “photoplethysmographic (PPG) signal” is a plethysmographic signal obtained using optical instruments.
A “videoplethysmographic (VPG) signal” is a plethysmographic signal extracted from video.
A “subject” is a living being. One example subject 100 is shown in
A “video”, as is generally understood, is a time-varying sequence of image frames acquired by a video imaging device. The video may contain other components such as, audio, time reference signals, frame rate, and the like.
A “video imaging device” is a single-channel or a multi-channel device for capturing video.
“Receiving image frames” is intended to be widely construed and includes: retrieving, capturing, acquiring, or otherwise obtaining image frames for processing to obtain a VPG signal for the subject. The image frames can be retrieved from a memory or storage device of the video imaging device or retrieved from a media such as a CDROM or DVD. Image frames can be obtained from a remote device over a network or downloaded from a web-based system or application which makes image frames available for processing. The image frames may be processed to compensate for motion induced blur, imaging blur, and slow illuminant variation. The video is preferably processed in overlapping batches of time-sequential image frames.
A “batch of image frames” refers to a plurality of time-sequential image frames.
“A region of exposed skin” refers to an unobstructed view of the subject's skin as seen through the lens of the video imaging device.
“Isolating pixels” in a region of exposed skin can be effectuated using techniques which include pixel classification, object identification, thoracic region recognition, color, texture, spatial features, spectral information, pattern recognition, face detection, facial recognition, and a user input. Methods for classifying pixels in an image are disclosed in several of the above-incorporated references by Wang et al. Pixels may be weighted, averaged, normalized, or discarded, as needed. Isolated pixels are processed to obtain a time-series signal.
A “time-series signal” is a signal that contains frequency components that relate to the subject's cardiac function. More specifically, the time-series signal contains the sum total of the relative blood volume changes in the blood vessels close to the skin surface within the isolated region of exposed skin. These arterial pulsations comprise a dominant component of the time-series signals. In one embodiment, a time-series signal is obtained by averaging all pixel values in the isolated region of exposed skin to obtain a channel average on a per-frame basis. A global channel average is computed, for each channel, by adding the channel averages across multiple image frames and dividing by the total number of frames comprising the batch. The channel average is then subtracted from the global channel average and the result is divided by a global channel standard deviation to obtain the time-series signal. The time-series signal may be filtered with a cutoff frequency defined as a function of a frequency of the subject's cardiac pulse. The time-series signal may be detrended to remove non-stationary components. Automatic peak detection may also be performed on the filtered signal. A VPG signal corresponding to the subject's cardiac function is extracted from the time-series signal.
“Extracting a continuous VPG signal” means to perform signal separation on the time-series signals to extract a continuous videoplethysmographic signal for the subject. Methods for extracting a plethysmographic signal from a time-series signal are disclosed in the above-incorporated references by Mestha and Mestha et al. Independent component analysis (ICA) can be used to recover VPG signals from a time-series signal. ICA is a decomposition technique for uncovering independent source signal components from a set of observations that are composed of linear mixtures of underlying sources, i.e., independent components of the observed data. Constrained source separation is an independent component analysis method for separating time-series signals into additive sub-components using a reference signal as a constraint. In one embodiment, the reference signal has a frequency range that approximates a frequency range of the subject's cardiac pulse. Not all constraints can be used for constrained independent component analysis (cICA) because some constraints infringe classical ICA equivariant properties. Constraints that define or restrict the properties of the independent components should not infringe the independence criteria. Additional conditions can be incorporated using, for example, sparse decomposition of signals or fourth-order cumulants into the contrast function, to help locate the global optimum separating the components. The obtained signal is converted to zero-mean unit variance. A filtering step is performed in order to improve peak detection accuracy. HR signals can be filtered using, for example, a moving average filter with a suitable moving window of size N frames. One example moving average filter is given as:
where N is the number of frames in a moving window, x is the unfiltered plethysmographic signal, y is the filtered plethysmographic signal, n is the current frame i is the index designating the moving frame. Additional corrections may be necessary based on estimating the average of the amplitudes obtained from previous peaks. The VPG signal can also be filtered using, for example, an FFT-based phase preservation filter, a zero-phase digital filter, a linear time invariant (LTI) filter, a linear time varying (LTV) filter, a finite impulse response (FIR) filter, an infinite impulse response (IIR) filter, or a non-linear filter such as a median filter.
An “electrocardiogram (ECG) signal” (alternatively, EKG, from the Greek “kardia”, meaning heart) is a signal obtained from a sensor which senses the electrical activity of the heart.
“Receiving an ECG signal” is intended to be widely construed and includes: retrieving, capturing, acquiring, or otherwise obtaining an ECG signal for processing. The ECG signal can be retrieved from a memory or storage device of the ECG device or retrieved from a media such as a CDROM or DVD. ECG signals can be obtained from a remote device over a network or downloaded from a web-based system or application which makes such signals available for processing. In accordance with the teachings hereof, the VPG and ECG signals are used to determine the transit time of an arterial pulse pressure wave.
“Pulse transit time (PTT)” is the time it takes an arterial pulse pressure wave to travel between two points. An arterial pulse pressure wave is generated when the left ventricle of the heart contracts and pushes a volume of blood out the ascending aorta into the systemic arteries. The repeated push of this blood volume generates a pulsating wave. The PTT is therefore the time taken for the arterial pulse pressure wave, which is originated from the left ventricle, to propagate through the arterial network to the region of exposed skin where the VPG signal was obtained. As disclosed herein, an instantaneous PTT is determined by computing a phase difference, dø, (in radians per second) between two reference points on the temporally synchronous ECG and VPG signals, as given by:
PTT=dø/fHR
where fHR is the frequency of the subject's cardiac pulse (in beats per minute). One method for estimating a cardiac pulse rate from a signal extracted from video is disclosed in the above-incorporated reference by Kyal et al.
In one embodiment, the reference point on the ECG signal is the peak point of the R wave and the reference point on the VPG signal is the peak of the peripheral pulse. It should be appreciated that the reference point on the VPG signal can be any characteristic point such as, for example, a maximum or a minimum point on the VPG signal, an average point between a maximum and a minimum on the VPG signal, a maximum of first derivative of the VPG signal derivative, and a maximum of a second derivative of the VPG signal. The PTT obtained is an instantaneous PTT and the PTT is obtained by averaging the instantaneous PTT over several cycles. PTT can be used to determine blood pressure, blood vessel dilation over time, blood vessel blockage, and blood flow velocity. Furthermore, PTT can be used as an indirect marker for assessing the occurrence of cardiac arrhythmia, cardiac stress, heart disease, and peripheral vascular disease. Since movement during video acquisition can adversely impact the extracted VPG signal, movement should be compensated for. In various embodiments hereof, a threshold level is set for movement.
A “threshold for movement” is a level of movement during video acquisition to determine whether motion artifacts may have been introduced into the video which will adversely impact the quality of the VPG signal extracted from the image frames during the time when the movement is determined to have occurred. A determination can be made whether a movement occurred during video acquisition by, for instance, using a motion detector or by visually observing the subject. Moreover, a determination can be made during batch processing of image frames by analyzing the isolated pixels. Such a determination can be made by: determining whether a location of a center pixel in the isolated region has changed relative to a fixed position; determining whether a size of the isolated region has changed relative to a size of a region isolated in at least one previous frame; determining whether a shape of the isolated region has changed relative to a shape of a region isolated in at least one previous frame; determining whether a color of pixels in the isolated region has changed relative to pixel colors of at least one previous frame; or by identifying a residual from frame differencing. If the movement is determined to be above the threshold level set for movement then movement has to be compensated in the current batch of image frames or these image frames need to be discarded. The threshold for movement may be pre-set by the user. The threshold may be based on a type of motion or a source of motion (i.e., by the subject or by the environment) or the time the movement occurred. The threshold may be automatically adjusted in real-time or manually adjusted by a user/technician as the video of the subject is being captured by the video imaging device. The exact level of this threshold for movement will largely depend on the application where the teachings hereof find their intended uses. Therefore, a discussion with respect to a particular threshold level is omitted. Other responses to movement exceeding the threshold include: initiating an alert signal that movement is excessive; signaling a medical professional that movement has occurred; adjusting a position of the video camera; adjusting a position of the subject; changing a frame rate of the video imaging device; swapping the video imaging device for another video camera; moving a position of the video imaging device; and stopping video acquisition altogether.
“Processing”, as used herein, broadly includes the application of any mathematical operation applied to data, according to any specific context, or for any specific purpose.
Reference is now being made to the flow diagram of
At step 502, receive a video comprising a plurality of time-sequential image frames of a region of exposed skin of a subject where a videoplethysmographic (VPG) signal can be registered by at least one imaging channel of the video imaging device used to capture that video.
At step 504, receive an electrocardiogram (ECG) signal obtained using a sensor placed on the subject's body where an ECG signal can be obtained.
At step 506, process batches of image frames of the video to obtain a continuous VPG signal for the subject.
At step 508, determine a pulse wave transit time between two reference points on the VPG and ECG signals. In this embodiment, further processing stops.
It should also be appreciated that the flow diagrams depicted herein are illustrative. One or more of the operations may be performed in a differing order. Other operations may be added, modified, enhanced, or consolidated. Variations thereof are intended to fall within the scope of the appended claims.
Reference is now being made to
In
In the embodiment of
Movement Analyzer 609 continuously analyzes batches of image frames and makes a determination whether a movement occurred which exceeds a threshold level set of movement. A user may set the threshold level for movement on the smartphone prior to video acquisition and ECG signal capture, or may adjust the threshold dynamically during video and ECG signal acquisition by the smartphone. Adjustments can also be dynamically made by the user of the smartphone to the size of batches being processed and communicated to Batch Processor 607 so that a next batch of image frames can be processed with the dynamically adjusted batch size.
PTT Processor 610 receives the VPG signal and the ECG signal 605 and determines the transit time for the arterial pulse wave. Central Processor (CPU) 611 retrieves machine readable program instructions from Memory 612. The CPU and Memory, alone or in conjunction with other processors and memory, may be configured to assist or otherwise facilitate the functionality of any of the modules of system 606.
Processing system 606 is shown in communication with a workstation 613. A computer case of the workstation houses various components such as a motherboard with a processor and memory, a network card, a video card, a hard drive capable of reading/writing to machine readable media 614 such as a floppy disk, optical disk, CD-ROM, DVD, magnetic tape, and the like, and other software and hardware needed to perform the functionality of a computer workstation. The workstation further includes a display device 615, such as a CRT, LCD, or touchscreen device, for displaying information, video image frames, VPG signals, ECG signals, computed values, medical information, results, and the like. A user can view any of that information and make a selection from menu options displayed thereon or directly from the smartphone. Keyboard 616 and mouse 617 effectuate a user input or selection. The workstation implements a database in storage device 618 wherein patient records are stored, manipulated, and retrieved in response to a query. Such records, in various embodiments, take the form of patient medical history stored in association with information identifying the patient along with medical information. Although the database is shown as an external device, the database may be internal to the workstation mounted, for example, on a hard disk therein.
It should be appreciated that the workstation of
Although shown as a desktop computer, it should be appreciated that the workstation can be a laptop, mainframe, or a special purpose computer such as an ASIC, circuit, or the like. The embodiment of the workstation of
It should be appreciated that some or all of the functionality performed by any of the modules or processing units of system 606 can be performed, in whole or in part, by the workstation placed in communication with the smartphone 601 over network 619. It should be understood that any of the functionality performed by the processing system 606 and/or the workstation 613 may be performed, in whole or in part, by the smartphone 601. The embodiment shown should not be viewed as limiting the scope of the appended claims strictly to that configuration. Various modules may designate one or more components which may, in turn, comprise software and/or hardware designed to perform the intended function.
A single-lead ECG signal was recorded for a subject using a sensor with a sampling rate of about 500 Hz. At the same time, video was captured of a facial region of the subject using a video camera with a capability of recording 120 frames per second. The ECG sensor was distal to the facial region. Image frames of the captured video were processed and a VPG signal extracted.
It will be appreciated that the above-disclosed and other features and functions, or alternatives thereof, may be desirably combined into other different systems or applications. The teachings hereof can be implemented in hardware or software using any known or later developed systems, structures, devices, and/or software by those skilled in the applicable art without undue experimentation from the functional description provided herein with a general knowledge of the relevant arts.
One or more aspects of the methods described herein are intended to be incorporated in an article of manufacture which may be shipped, sold, leased, or otherwise provided separately either alone or as part of a product suite or a service. Presently unforeseen or unanticipated alternatives, modifications, variations, or improvements may become apparent and/or subsequently made by those skilled in this art which are also intended to be encompassed by the following claims.
The teachings of any publications referenced herein are each hereby incorporated by reference in their entirety.
