Patent Application
20170188868
This application claims the priority benefit of Provisional Application Nos. 62274765, 62274761, 62274763, 62274766, and 62274770, each of which were filed on Jan. 4, 2016, the entire disclosure each are incorporated herein by reference.
The embodiments herein relate generally to cardiopulmonary health monitoring and more particularly to analysis software combined with transducers to capture multi-channel vibration signals along with an electrocardiogram signal for the measurement of heart functions.
Heart disease is the leading cause of death accounting for more than one-third (33.6%) of all U.S. deaths. Overall cardiac health can be significantly improved by proper triage. Low invasive and non-invasive ultrasound techniques (e.g. echocardiogram) are standard procedures, but the requirement of expensive devices and skilled operators limit their applicability. The following are the various types of heart disease that can be diagnosed and treated using the separated signal, namely, Coronary artery disease, Heart murmurs and valve abnormalities, Heart failure, Heart rhythm abnormalities (arrhythmias), Vascular disease, congenital heart disease, Cardiac resynchronization and Risk factor modification. A physician can work with patients to perform a comprehensive evaluation and design a personalized plan of care aimed at keeping them healthy.
The cardio pulmonary system which consists of the respiratory components, snoring components, and cardiac components, creates vibrations during each cardiac cycle. The vibrations are the result of the lung sounds, heart sounds, acceleration and deceleration of blood due to abrupt mechanical opening and closing of the heart valves during the cardiac cycle.
The exemplary embodiments herein provide a method and system based on a technique to identify the separated cardiopulmonary signals and to extract information contained in vibration objects. In some embodiments, machine learning, auditory scene analysis, or spare coding are used as approaches to the source separation problem. Data is obtained using a tri-axial accelerometer or multiple tri-axial accelerometers placed on different points of a torso. Further note that the techniques and methods herein are not limited to acoustic, electrical or vibrational data as might be used in some stethoscopes, but can also be applied to other forms of monitoring such as echo imaging or sonograms, magnetic resonance imaging (MRI), computed tomography (CT) scanning, positron emission tomography (PET) scanning, and monitoring using various forms of catheterization. The techniques and methods herein are primarily applicable to monitoring of heart valve events, but can be alternatively applied to other types of involuntary biological signaling emanating from the brain, intrauterine, pre-natal contractions, or elsewhere within both humans and other species.
Examples of cardiac vibration objects are the first sound, the second sound, the third sound, the fourth sound, ejection sounds, opening sounds, murmurs, heart wall motions, coronary artery sounds, and valve sounds of the Mitral valve opening and closing, Aortic valve opening and closing, Pulmonary valve opening and closing, Tricuspid valve opening and closing. Examples of the pulmonary vibration objects are the respiratory lung sounds, breathing sounds, tracheobronchial sounds, vesicular sounds, Broncho vesicular sounds, snoring sounds. A portion of the energy produced by these vibrations lies in the infra-sound range, which falls in the inaudible and low sensitivity human hearing range. For example, the vibration objects from the Mitral, Tricuspid, Aortic, and Pulmonary valve openings fall in a lower range of vibrations such as 0 to 60 Hertz, whereas vibration objects from the Mitral, Tricuspid, Aortic, and Pulmonary valve closings fall in a higher range of vibrations such as 50 to 150 Hertz. A portion of the energy produced by these vibrations falls in the audible hearing range. Accelerometer transducers placed on the chest capture these vibrations from both these ranges.
Source identification analysis in accordance with the methods described herein identify individual vibration objects described above from the source separated vibration signals. The individual vibration signals are identified to be from the mitral valve, aortic valve, tricuspid valve, the pulmonary valve, coronary artery, murmurs, third sound, fourth sound, respiratory sound, breathing, and snoring during individual heart beats. The identified signals are marked to indicate their start with respect to the start of the electrocardiogram or EKG.
The exemplary embodiments may be further understood with reference to the following description and the appended drawings, wherein like elements are referred to with the same reference numerals. The exemplary embodiments describe a system and method of source identification of the individual cardiopulmonary signals from the composite vibrations captured on the chest wall. Specifically, psychoacoustics are considered in identifying the cardiac vibration signals captured through the transducers. The system, the psychoacoustics, and a related method will be discussed in further detail below.
The embodiments can include different source identification techniques specifically used for tagging individual cardiopulmonary signals for application in a non-linear time variant system, such as Principal component analysis, Gabor filtering, Generalized Cross Correlation (GCC), Phase transform (PHAT), ROTH, SCOT and Band Filtering. The techniques herein can use one or more among 1) Spectral information, 2) Relations among channels, and 3) Relations among events in the form of relative times of occurrence.
The exemplary embodiments provide a novel approach for small, portable, robust, fast and configurable source separation based software with transducer hardware 103, 203. The use of the vibration signal pattern and novel psychoacoustics help bypass conventional issues faced by linear time invariant systems.
The following are the various types of heart disease that can be diagnosed and treated using the identifies signals, namely, Coronary artery disease, Heart murmurs and valve abnormalities, Heart failure, Heart rhythm abnormalities (arrhythmias), Vascular disease, congenital heart disease, and Risk factor modification. A physician can work with patients to perform a comprehensive evaluation and design a personalized plan of care aimed at keeping them healthy.
The exemplary embodiments of the system and method proposed here are shown in
The exemplary embodiments of the system and method proposed here for the source identification of the cardiopulmonary signals 200 are shown in
The exemplary embodiments of the system and method proposed here for the source identification of the cardiopulmonary signals from the composite signal 300 are shown in
The exemplary embodiments of the system and method proposed here draw inspirations from biology with respect to the cardiac cycle in-relation with electrocardiogram and accelerometer transducer captured cardiac signal. A timeline chart 400 in
The exemplary embodiments of the system and method proposed here provide a source identification analysis algorithm that allows for the tagging of the vibrations from the cardiopulmonary system. The sensors can be modeled as:
x
1(t)=s1(t)+n1(t)
x
2(t)=αs1(t+D)+n2(t),
Where x are the sensor signals and s and n are the sources and the noise respectively (assumed to be uncorrelated). A system 500 can include filtering or weighting (using transforms 501 (H1) and 502 (H2)) of the respective input sensor signals X1 and X2. Cross correlating the filtered sensor signals (using blocks 504, 505, and 506) and finding the peak using peak detector 507 will result in the estimate of the delay D as can be seen in the block diagram of
The chart 600 of
One of the proposed embodiments of the source identification is using Gabor analysis on the source separated signals. After the signals have been separated the next step is to identify them, this process is usually known as source tagging. The need to do source tagging arises from the permutation problem caused by the source separation algorithm. The separated sources obtained by the source separation are similar to the original sources up to a scale and a permutation factor.
In another embodiment, the source can be identified by manually tagging them against a synchronized EKG signal and taking advantage of the timings relative to the QRS wave. This way, however is usually slow and time consuming and an automatic tagging algorithm is thus preferable. Since the different heart sounds comes from different locations in the heart, it is expected that each source will have a unique phase relation between the sensors that are located at fixed points during the data gathering phase. This phase relation can be made evident by using signal representation with a dictionary suited for highlighting frequency and phase relations with a greedy algorithm such as the matching pursuit algorithm. One of the proposed algorithms herein performs a Gabor analysis in the source separated signals using a finite Gabor Dictionary of fixed frequencies and with variable phase delay. The system finds the delay that minimizes the reconstruction error for each source and creates a group of features that is incorporated into a decision-making and classification algorithm. The classification algorithm will combine the features extracted by the Gabor analysis with other features that comes from the Principal Component Algorithm (PCA) and cross correlation analysis that uses a set of manually tagged patient's tracks as training. The system is composed of three modules or stages: A suitable Gabor dictionary is created that can serve as the basis representation of the signals. An optimization algorithm aims to find the delay that will minimize the reconstruction error in a giving delay range. Collection and organization of all the features extracted from the second stage. For each one of the stages different techniques were tried in order to achieve the best results. The Gabor dictionary selected has the function:
In the equation 8 is the delay, a is the Gaussian decay of the atom and f is the frequency of operation of the Gabor. The system initially searches for the Gabor atoms that best represent the signal by doing a sweep in frequency and selecting those Gabor atoms that produced a larger weight matrix after the matching pursuit. The system was subsequently changed to a group of fixed frequency Gabor atoms exploiting the fact that for the M1, T1, A2 and P2 sounds vibration reside mainly in the range of 50-300 Hz and the Aortic opening vibration resides mainly in the low frequency range (from 0-50 Hz). The fixed Gabor approach sacrifices a little bit of reconstruction error in favor of a faster computation and easier analysis since the frequencies were fixed. The first optimization technique used was the, Gradient Descent algorithm: A first-order optimization algorithm that looks for the minimum of a given function by taking steps proportional to the negative of the gradient. The system is very dependent on the initial guess and number of minima of the system. A second approach was using the minimize function, which uses the Polack-Ribiere algorithm of conjugate gradients to compute search directions, and a line search using quadratic and cubic polynomial approximations. The system is also dependent on the initial value of the search but is more efficient in choosing the parameters to find the minimum. Finally, a brute force approach was implemented that sweeps the signal with different delays and then selects the one that resulted in a minimum function. In order to improve speed on this algorithm, a broad search is done first and then a refined search is done around the minimum point found by the broad search.
The exemplary embodiments of the system and method proposed here provide a source identification algorithm for the vibrations from the cardiopulmonary system. Referring to the chart 700 of
The exemplary embodiments of the system and method proposed here provide a source marking algorithm for the vibrations from the cardiopulmonary system. Next step is to use the PCA to determine which source is associated with which event (Mitral closing & opening, Tricuspid closing & opening, Aortic opening & closing, Pulmonic opening and closing). The architecture for automatic source tagging and timing of valvular events is further described. One way to identify which events are relevant to a source is by manually tagging the sources against the synchronized EKG signal and taking advantage of the timings relative to the QRS wave (identification of the S1 and S2 sounds using the EKG signal as the reference has been widely researched in studies. Another approach is an automatic tagging algorithm. The tagging is composed of a classifier preceded by a feature extraction algorithm. For the timing, the system can exploit the computations of one of the feature extraction algorithms to obtain an energy contour from which the time location of a given event can be inferred. Because the embodiments herein build upon having the ability to capture the signal at different locations simultaneously, to the system exploits the relations among channels to extract additional information about the sources. Likewise, since some source separation algorithms where channels relations are associated with location, the system can leverage on the intrinsic relations among the channels to extract relevant information that helps distinguish among the events. The system herein hypothesizes that phase information between channels is relevant for distinguishing among cardiac events since valves are located at different positions within the heart. Perhaps, one of the most distinctive features of the cardiac events is their relative order of occurrence, which repeats periodically with each heartbeat. Time information extracted from the set of sources can be utilized to localize the occurrence of each source signal within the heart cycle. Therefore, the features proposed here are conceived to provide three aspects: 1) Spectral information, 2) Relations among channels, and 3) Relations among events in the form of relative times of occurrence. In some embodiments, the system includes a feature extraction algorithm based on multichannel Gabor basis decomposition using matching pursuit, and a second algorithm that uses eigen-decomposition of a covariance matrix extracted from the source signals to obtain a cross correlation function between channels. For the Gabor basis, consider the reconstruction of a 2-channel signal x (t). In this approach to the system can extract a phase relation between the sensors which are located at standard aortic and pulmonic auscultation positions. As mentioned above, the system is interested on features that reflect spectral content as well as the channel interrelations. In the two channel case, each basis function is a pair of Gabor functions with equal frequency f and envelope width h and a phase difference θ. The tunable parameter θ can be swept across a range of values for which the matching pursuit reconstruction error can be obtained. The behavior of the reconstruction error for a source signal over different values of θ provides information about the phase different between the two channels for the particular source signal. In particular, the value of θ that attains the minimum error within the defined range of values is taken as the optimal channel delay, and the time average of the activation coefficients obtained by matching pursuit at the optimal θ provide and spectral characterization of the source. For the feature Extraction by Self-Similarity Template Based-on PCA, we define a self-similarity using the concept of eigenfilter. Let yi the a two-channel source signal defined in (1), and let Yi its discretized version of size Nx2. For filter of length Nw, we compute the tap-delay matrices Δ(j). The joint-channel eigenfilter correspond to the leading right-singular vector u of the tap-delay centered matrix Δi obtained by removing the mean of each column of Δi. The eigenfilter u can be split into the first and second channel components. The first channel component u(1) corresponds to the first Nw entries of u, and u(2) to the remaining entries going from index Nw+1 to last index 2Nw. The proposed feature is the cross-correlation function between u(1) and u(2). Note that this function contains the main frequency components of the source signals expressed in the time domain and the peak provides information about the channel relations. For different delays between the two channels the cross correlation peaks at different locations, accordingly. To extract relative time information contains cues for the classification of the cardiac events, we adopt a simple approach that uses the energy contours. The process consists of three basic steps: Compute energy contours for all source signals, Compute timings for each source signal, Compute features of source i by averaging the sum of the remainder sources centered at the timings. The energy of each source signal Yi is calculated using the leading, quadrature pair of right singular vectors of their corresponding tap-delay matrix. For classification, a set of training exemplars (sources) have been manually tagged with the respective events. Each source signal is then represented by the feature vectors described above. Tests have been carried out on pooled covariance linear discriminant analysis, quadratic discriminant analysis, k-nearest neighbours and support vector machines both linear kernel and Gaussian kernel. The features described in the previous sections can be used in different manners for tagging, for example: Tagging using only Gabor-based features, Tagging using only self-similarity features, Tagging using only time-based features, Tagging using all Gabor-based, self-similarity, and time-based features. There are even sub cases of the situations considered above, For instance, the combination of Gabor and self-similarity features can be done on a single classifier or using ensembles. The first validation examples only address the cases in the bullet items. In addition to the classifier, preprocessing of the feature vectors is also performed. A centering vector xcent is calculated using the mean over the training set xtrain. This step is followed by a linear transformation based on PCA to produce a much smaller dimension feature vector. The preprocessing is summarized in Algorithm 1, in
The exemplary embodiments of the system and method proposed here provide a source marking algorithm that allows from the explanation earlier for the marking of the Mitral valve closing (MC), Mitral valve opening (MO), Aortic valve opening (AO), Aortic valve closing (AC), Tricuspid valve closing (TC), Tricuspid valve opening (TO), Pulmonary valve closing (PC) and Pulmonary valve opening (PO) signals. The extracted individual valve vibration objects are aligned into a signal for each of the four valves across multiple heart beats. The chart 800 in
The exemplary embodiments of the system and method proposed here provide a source marking algorithm for the vibrations from the cardiopulmonary system, using information about the time of occurrence of the event. Automated finding of M1, T1, A2 & P2: After going through the timing plots of different patients, it was decided to first determine A2 and P2 timing and their corresponding sources and then find M1 and T1 from the rest of sources. Automatic A2 & P2 finding: 1) The number of zero-crossings is found for each SS source. Then the noisy source that is associated with the maximum zero-crossing is discarded. 2) Automated calculation of QRS onset points for all heartbeats in each source. 3) Automated calculation of beginning and ending points of all vibrations in each source. 4) Applying PCA approach on all sources that outputs the time difference between QRS onset and peak of PCA signal (timing vector) as well as delay between aortic and pulmonic channels. 5) Based on probability density estimation of each source's timing vector, strong peak(s) that correspond(s) to timing(s) more than 300 ms are accepted as A2 and P2 candidates and those sources that don't satisfy this condition are discarded. Also new candidate timings are sorted in ascending order. 6) Variation, length of samples more than 300 ms (L) and some other measurements are calculated from accepted timing vectors of accepted sources and are stored in a structure for further analysis. 7) If there is only one strong peak in probability density estimation plot whose timing is more than 300 ms, then a variable called “ind_both” is set to 1. If there are two or more strong peaks whose timing is more than 300 ms, if max peak's timing is over 300 ms, then “ind_both” variable is set to 1 but if max peak's timing is lower than 300 ms but another strong peak's timing is more than 300 ms, then “ind_both” is set to 0. If there are two or more sources whose “both_indic” variables are equal to 1, then “both_indic”=1.8) In case of having no valid timing, A2 and P2 are set to zero. 9) In case of having one valid timing, if L>6 then it is marked as A2 and its corresponding source number is also saved. P2 is set to zero. 10) In case of having two valid timings:
if both_indic=1,
if L1>=6 samples, then A2 is set to first timing in sorted timing vector Else A2=0
if A2=0 and L2>=6, then A2 is set to second timing in sorted timing vector Else P2=0
if A2˜=0 and L2>=6, then P2 is set to second timing in sorted timing vector Else P2=0
if both_indic˜=1,
if L1>=4 samples & V1<=800 then A2 is set to first timing in sorted timing vector Else A2=0
if A2=0 and L2>=4 & V1<=800, then A2 is set to second timing in sorted timing vector Else P2=0
if A2˜=0 and L2>=6 & V2<=800 then P2 is set to second timing in sorted timing vector Else P2=0
11) In case of having more than two valid timings,
if both_indic=1
for loop: if L1>=6 & V1<=800, then A2 is set to current timing in sorted timing vector; break Else A2=0; End of for loop
if A2=0, then P2=0;
if A2˜=0 & last timing in the sorted timing vector
if L>=6 & V1<=800, then P2 is set to last timing in the sorted timing vector
if A1˜=0 & there are two timings after A1 timing,
if V2<V1, then P2 is set to second timing after A2.
Else P2 is set to first timing after A2.
if A1˜=0 & there are more than two timings after A1 timing,
if L1>=6 & V1<=800, then P2 is set.
if both_indic ˜=1,
for loop: if L1>=8 samples & V1<=1000 then A2 is set to current timing in sorted timing vector Else A2=0; End of for loop.
if A2=0, then P2=0;
if A2˜=0
for loop: if L>=8 & V1<=1000,
then P2 is set to current timing in the sorted timing vector; break
Else P=0;
End of for loop
At the end, A2 and P2 timings and their corresponding source numbers can be saved in an excel spreadsheet along with the patient name.
Automatic M1 & T1 finding:
if L1>=6, then M1 is set to first timing in the sorted timing vector Else M1=0
if M1=0 & L2>=6, then M1 is set to second timing in the sorted timing vector and T1=0
if M1˜=0 & L2>=6 and second timing is less than double the first timing
then T1=second timing in the sorted timing vector Else T1=0
if both_indic ˜=1
L1>=4 & V1<=800, then M1 is set to first timing in the sorted timing vector Else M1=0
If M1=0 & L2>=4 & V2<=800, then M1=second timing in the sorted timing vector Else T1=0
If M1˜=0 & L2>=4 & V2<=800, then T1=second timing in the sorted timing vector Else T1=0
11) In case of having more than two valid timings,
if both_indic ˜=1,
for loop: if L>=8 & V<=1000, then M1 is set to current timing in the sorted timing vector Else M1=0; break; end of for loop
if M1=0, then T1=0;
for loop: if M1˜=0 & L>=6 & V<=1000, then T1 is set to current timing in the sorted timing vector; break, end of for loop
if M1˜=0 & L2>=6 and second timing is less than double the first timing
then T1=second timing in the sorted timing vector Else T1=0
if both_indic=1
low_var=Find sources with variations less than 15.
From these low_var sources, find those whose length are more than 9 samples (len_low_var).
If length of low_var>=2 & len_low_var>=2
M1 is set to the timing corresponding to the lowest variation
T1 is set to the timing corresponding to the second lowest variation
for loop: If L>=5 & V<800, then M1 is set to the current timing in the sorted timing vector Else M1=0
If M1=0, then T1=0
for loop:
If there is only one timing after M1 & L>=5 & V<=800, then T1=current timing in the sorted timing vector; break
If there are two timings after M1,
if V1<300, then T1=first timing after M1 in the sorted timing vector; break
Else if V2<V1, then T1=second timing after M1 in the sorted timing vector; break
Else, T1=first timing after M1 in the sorted timing vector; break
If there are more than two timings after M1,
if V1<5, then T1=current timing after M1 in the sorted timing vector; break
Else if V2<V1, then T1=current timing after M1 in the sorted timing vector; break
Else, T1=current timing after M1 in the sorted timing vector; break
If there is only on candidate source:
set max's peak timing as ref time, remove samples with diff more than 50 ms relative to ref time and update the AO candidate and its variation.
III. If time diff btw AO candidate and max peak's timing >10 ms, find a qualified peak and its corresponding timings that is closest to AO candidate, set that peak's timing as ref time, remove samples with diff more than 50 ms relative to ref time and update the AO candidate and its variation.
If there are two candidate sources:
The following steps from the first case are implemented:
If there are more than two candidate sources:
The following steps from the first case are implemented:
To improve the results in the exemplary embodiment the system can perform AO detection using a new PCA approach such as: Mean removal, Min-Max removal, or Different window length for correlation. Detailed analysis on delay calculation from cross-correlation template. Source tagging using PCA features and different classifiers on both tagged and untagged tracks for both High Pass and Low Pass. Defined two timing calculation algorithm 1) with threshold 2) without threshold and got the timing distributions for different test cases. To improve AO detection using new PCA algorithm, a few different tests were done such as: Removed the mean of each source's template (X matrix) before calculating covariance matrix. For some sources, cross-correlation template shows zero delay at a discontinued point although there is a delay between peaks of templates. Removed min-max of eigenvector before performing cross-correlation showing a continuous behavior with a delay. Then ran different SSs on the same patient to see if these observations are consistent. There are different scenarios when calculating the delay such as: Maximum peak is positive or negative, The Delay of Max peak is positive or negative. The approach herein is to select positive maximum peak with the positive delay. The goal in some embodiments is to tag or label the sources with an event (HP: M1, T1, A2, P2, LP: AO). The cross-correlation signals generated by new PCA algorithm are used as features to train the classifiers and then tag sources of a test signal. A similar approach can be extended for the identification of other cardiopulmonary signals, like coronary artery sounds, snoring, murmurs, respiration sounds in one embodiment.
The exemplary embodiments develop a novel method of source identification, which in one embodiment using the Pulmonary and Aortic, and in addition possibly the Tricuspid and Mitral auscultation locations, lends the system capable of identifying individual cardiopulmonary events from the vibrations.
The exemplary embodiment of system and method described is the development on an embedded hardware system, the main elements required to capture body sounds are the sensor unit that captures the body sounds, digitization, and digital processing of the body sounds for noise reduction, filtering and amplification. Of course, more complicated embodiments using the techniques described herein can use visual sensors, endoscopy cameras, ultrasound sensors, MRI, CT, PET, EEG and other scanning methods alone or in combination such that the monitoring techniques enable improvement in terms of source separation or identification, and/or marking of events such as heart valve openings, brain spikes, contractions, or even peristaltic movements or vibrations. Although the focus of the embodiments herein are for non-invasive applications, the techniques are not limited to such non-invasive monitoring. The techniques ultimately enable diagnosticians to better identify or associate or correlate detected vibrations or signaling with specific biological events (such as heart valve openings and closings, brain spikes, fetal signals, or pre-natal contractions.)
It will be apparent to those skilled in the art that various modifications may be made in the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the method and system described and their equivalents.
| Number | Date | Country | |
|---|---|---|---|
| 62274765 | Jan 2016 | US | |
| 62274761 | Jan 2016 | US | |
| 62274763 | Jan 2016 | US | |
| 62274766 | Jan 2016 | US | |
| 62274770 | Jan 2016 | US |
