Patent Application
20230397887
The disclosure relates to the technical field of signal processing, in particular to a scatter diagram classification method and device for photoplethysmography (PPG) signal.
A time interval between peak points of adjacent ECG signal waveforms is regarded as a cardiac cycle length, which is called an inter-beat interval. An inter-beat interval scatter diagram (simply referred to as scatter diagram) is a two-dimensional coordinate way to identify corresponding heart rhythm characteristics by observing the data change of the inter-beat interval. The abscissa of each scatter point in the scatter diagram represents a previous inter-beat interval of a certain heart beat, and the ordinate represents a next inter-beat interval of a certain beat. By identifying the distribution regulation of scatter points in the scatter diagram, the overall dominant rhythm of the heart can be known. An RR interval scatter diagram can be used to evaluate heart rate fluctuation, autonomic nervous regulation and heart rate variability, and can also be used to diagnose arrhythmia and evaluate the prognosis of diseases. When analyzing heart rhythm characteristics by using an inter-beat interval scatter diagram, the larger the number of scatter points, the more obvious a formed typical image, and the more accurate an analysis result, which means that a long time of continuous acquisition (for example, at least half an hour) is required. However, a conventional ECG signal acquisition method is not suitable for long-time monitoring of a test object at any time.
Photoplethysmography (PPG) is a non-invasive method to detect the change of blood volume in viable tissue by photoelectric means. Cardiac impulses make the blood flow per unit area in the blood vessel change periodically, causing blood volume to change accordingly, so that a PPG signal reflecting the amount of light absorbed by blood will also change periodically, and the periodic change of the PPG signal is closely related to cardiac impulses and the blood pressure change. For PPG signals, a time interval between the maximum peak points of adjacent signal waveforms is also equal to an inter-beat interval.
The purpose of the present disclosure is to provide a scatter diagram classification method and device for a photoplethysmography (PPG) signal, electronic equipment, a computer program product and a computer-readable storage medium to overcome the defects of the prior art. By extracting inter-beat interval data from PPG signals to generate a scatter diagram, and then introducing the scatter diagram into an artificial intelligence network for confirming the type of the scatter diagram for type confirmation, so as to reduce the scatter diagram generation difficulty, and enrich the application scenarios of PPG in the field of health monitoring.
In order to achieve the above object, the first aspect of the embodiment of the disclosure provides a scatter diagram classification method for a PPG signal, comprising:
Preferably, performing scatter point two-dimensional coordinate preparation processing according to the PPG sampling signal to generate a scatter point two-dimensional coordinate sequence specifically comprises:
Preferably, performing scatter diagram resolution confirmation processing according to the sampling frequency and a preset maximum value of inter-beat interval to generate a scatter diagram resolution specifically comprises:
Preferably, performing scatter diagram initialization processing according to the scatter diagram resolution to generate a scatter diagram two-dimensional tensor specifically comprises:
Preferably, performing scatter point marking processing on the scatter diagram two-dimensional tensor according to the scatter point two-dimensional coordinate sequence specifically comprises:
Preferably, using a convolutional neural network of an artificial intelligence network to perform multilayer convolution pooling calculation on the scatter diagram two-dimensional tensor to generate a four-dimensional output tensor specifically comprises:
Preferably, using the fully connected neural network of the artificial intelligence network to perform multilayer full connection calculation on the four-dimensional output tensor to generate a two-dimensional output tensor specifically comprises:
Preferably, using the normalization processing layer of the artificial intelligence network to perform normalization index calculation on the two-dimensional output tensor to generate a normalization two-dimensional tensor specifically comprises:
Preferably, performing classification confirmation processing according to the normalization two-dimensional tensor to generate confirmation data (the confirmation data comprising classification reasonable information and classification unreasonable information) specifically comprises:
A second aspect of the embodiment of the disclosure provides a scatter diagram classification device for a PPG signal, comprising:
A third aspect of the embodiment of the disclosure provides electronic equipment, comprising a memory, a processor and a transceiver.
The processor is configured to be coupled with the memory, and read and execute instructions in the memory, so as to realize the method steps in the first aspect.
The transceiver is coupled with the processor, and the processor controls the transceiver to send and receive messages.
A fourth aspect of the embodiment of the disclosure provides a computer program product, which comprises a computer program code that, when executed by a computer, causes the computer to perform the method described in the first aspect.
A fifth aspect of the embodiment of the disclosure provides a computer-readable storage medium, which stores computer instructions that, when executed by a computer, cause the computer to execute the method described in the first aspect.
The embodiment of the disclosure provides a scatter diagram classification method and device for a PPG signal, electronic equipment, a computer program product and a computer-readable storage medium. By extracting inter-beat interval data from PPG signals to generate a scatter diagram, and then introducing the scatter diagram into an artificial intelligence network for confirming the type of the scatter diagram for type confirmation, therefore the scatter diagram generation difficulty is reduced, and the application scenarios of PPG in the field of health monitoring are enriched.
In order to make the object, technical solution and advantages of the disclosure more clear, the invention will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of them. Based on the embodiments of the disclosure, all other embodiments obtained by those skilled in the art without creative labor are within the scope of the disclosure.
Before the embodiment of the disclosure is described in detail, the structure and data formats of an artificial intelligence network for confirming scatter diagram classification mentioned above are briefly described.
After the artificial intelligence network performs corresponding classification confirmation calculation on a scatter diagram, two weight data are obtained: the classification reasonable weight data and the classification unreasonable weight data. The embodiment of the present disclosure finally obtains a classification confirmation result (reasonable classification and unreasonable classification) from the two weight data. For example, when the artificial intelligence network is specifically an atrial fibrillation scatter diagram confirmation network, atrial fibrillation scatter diagram classification confirmation calculation is performed for the current scatter diagram. When the classification confirmation result is reasonable classification, it is considered that the current scatter diagram is an atrial fibrillation scatter diagram, and when the classification confirmation result is unreasonable classification, it is considered that the current scatter diagram is not an atrial fibrillation scatter diagram.
The artificial intelligence network here is composed of a convolutional neural network, a fully connected neural network and a normalization processing layer. The convolutional neural network is composed of a plurality of convolutional network layers, and the fully connected neural network is composed of a plurality of fully connected layers.
The convolutional neural network is used to perform convolution calculation and pooling calculation of the scatter diagram, and outputs feature data for further learning and calculation by other networks. The convolutional neural network is composed of a plurality of convolutional network layers, and the output data of each convolutional network layer will be used as the input data of the next convolutional network layer. Each convolutional network layer consists of one convolution layer and one pooling layer. Wherein, the function of the convolution layer is convolution operation, and the purpose of convolution operation is to extract the features of the input data. The convolution layer of the first convolutional network layer extracts some low-level features (such as edges, lines, and angles), and the convolution layers of subsequent convolutional network layers will continuously iterate from low-level features and extract more complex high-level features. The pooling layer has two functions: one is to keep translation, rotation and scale invariance, and there are two commonly used processing methods: mean-pooling and max-pooling; the other function is to reduce the number of parameters and calculation amount of the next convolution layer on the premise of keeping the main features of the output data of the previous convolution layer, so as to prevent over-fitting.
Here, in each convolutional network layer, the scatter diagram will perform convolution operation with a convolution kernel with a fixed size. Generally, the shape of the convolution kernel is 3×3, 5×5, and 7×7. After each convolutional network layer, the output data will have a shape change relative to the input data, but still keep the four-dimensional tensor form: the input four-dimensional tensor (for example, the shape is Pin4×Pin3×Pin2×Pin1), and the output four-dimensional tensor (for example, the shape is Pout4×Pout3×Pout 2×Pout1), wherein the Pin4, Pin3, Pin2 and Pin1 are four-dimensional, three-dimensional, two-dimensional and one-dimensional parameters of the input four-dimensional tensor respectively, and the Pout4, Pout3, Pout2 and Pout1 are four-dimensional, three-dimensional, two-dimensional and one-dimensional parameters of the output four-dimensional tensor respectively. After the calculation of each convolutional network layer, the shape of the output four-dimensional tensor is subjected to the following change in terms of dimensional parameters relative to the shape of the input four-dimensional tensor: (1) Pout4 vs Pin4, the four-dimensional parameter does not change and is always 1 in the embodiment of the present disclosure; (2) Pout3 and Pout2 vs Pin3 and Pin2, the three-dimensional parameter and the two-dimensional parameter change, which is related to the size of the convolution kernel and the setting of a sliding step of each convolution layer, as well as the size of a pooling window and a sliding step of the pooling layer; and (3) Pout1 vs Pin1, the one-dimensional parameter changes, which is related to a selected output space dimension (the number of convolution kernels) in the convolution layer.
An output result of the convolutional neural network will be input into the fully connected layer of the fully connected neural network for the full connection calculation. The fully connected layer is a complex network structure formed by a large number of neurons (nodes) connected to each other, and is some kind of abstraction, simplification and simulation of the organization structure and operation mechanism of the human brain. Each fully connected layer comprises a plurality of nodes, each node is connected with all nodes of the previous layer to synthesize the node data extracted from the previous layer for one node calculation, and the calculation result is taken as the value of the current node to be connected to and acquired by nodes of the next fully connected layer. The node calculation here is also called the full connection calculation. Here, the input data and output data formats of each fully connected layer are two-dimensional tensors: the input two-dimensional tensor (for example, in the shape of Qin2×Qin1) and the output two-dimensional tensor (in the shape of Qout2×Qout1), wherein Qin2 and Qin1 are two-dimensional and one-dimensional parameters of the input two-dimensional tensor, and Qout2 and Qout1 are two-dimensional and one-dimensional parameters of the output two-dimensional tensor. After the calculation of each fully connected layer, the shape of the output two-dimensional tensor is subjected to the following change in terms of dimensional parameters relative to the shape of the input two-dimensional tensor: (1) Qout2 vs Qin2, the two-dimensional parameter does not change and is always 1 in the embodiment of the present disclosure; and (2) Qout1 vs Qin1, the change of the one-dimensional parameter is related to the total number of nodes in the fully connected layer. Specifically, the total number of nodes of the last fully connected layer in the embodiment of the present disclosure is 2, and Qout1 is 2 accordingly.
It can be seen from the above that the output of the convolutional neural network is a four-dimensional tensor, and the input of the fully connected neural network is a two-dimensional tensor, so it is necessary to perform the reduce dimension processing on the output result of the convolutional neural network layer from the output four-dimensional tensor shape (Pout4×Pout3×Pout2×Pout1) to the input two-dimensional tensor shape (Qin2×Qin1) of the fully connected neural network. The two-dimensional tensor shape finally output by the fully connected neural network should be 1×2, and the two data in the two-dimensional tensor are the two weight data obtained after the artificial intelligence network performs classification confirmation calculation on the scatter diagram: the classification reasonable weight data and the classification unreasonable weight data.
After the fully connected neural network outputs the two weight data, the normalization processing layer is used to normalize the weight data. Specifically, the normalization processing layer will use a Softmax function (the normalization exponential function) to perform normalization exponential calculation on the two weight data, and convert the two weight data into two probabilities less than 1, so as to facilitate subsequent classification confirmation judgment. Here, the two probabilities are: classification reasonable probability and classification unreasonable probability, and the sum of the two probability data should be 1.
Finally, the embodiment of the present disclosure will select the one with a larger value from the two probabilities as a classification confirmation result, that is, if the one with a larger value is the classification reasonable probability, then the classification result is reasonable classification, and if the one with a larger value is the classification unreasonable probability, then the classification result is unreasonable classification.
According to the scatter diagram classification method for a PPG signal provided by the first embodiment of the present disclosure, inter-beat interval data are extracted from PPG signals to generate a scatter diagram, then the scatter diagram is introduced into an artificial intelligence network for confirming the type of the scatter diagram for type confirmation, and finally the classification confirmation result is obtained according to the weight data output by the artificial intelligence network. As shown in
Step 1, acquiring the PPG signal.
Specifically, the equipment may acquire a real-time PPG signal of a test object through a local PPG signal acquisition device, and may also acquire the real-time PPG signal of the test object through a PPG signal acquisition device of other equipment connected to the equipment itself. And may also acquire the historical PPG signal of the test object from a local storage medium or a storage medium of other equipment connected to the equipment itself.
Here, the equipment is specifically a terminal equipment or a server for implementing the method provided by the embodiment of the present disclosure.
For example, the acquired PPG signal is a real-time PPG signal with a length of 30 minutes.
Step 2, performing signal sampling processing on the PPG signal according to a preset sampling frequency to generate a PPG sampling signal.
Here, the sampling frequency is used to sample the PPG signal, and the sampling frequency is stored in a local storage medium of the equipment. The PPG sampling signal, as shown in the diagram of a PPG sampling signal provided in the first embodiment of the disclosure in
For example, if the length of the PPG signal is 30 minutes and the sampling frequency is 250 Hz, the length of the PPG sampling signal is also 30 minutes, and there are 250*60*30=450,000 signal points.
Step 3, performing scatter point two-dimensional coordinate preparation processing according to the PPG sampling signal to generate a scatter point two-dimensional coordinate sequence.
Specifically including the following steps: step 31, sequentially extracting time information corresponding to maximum amplitude signal points of signal waveforms from the PPG sampling signal to generate peak time data, and forming a peak time data sequence from the peak time data,
wherein the number of the peak time data of the peak time data sequence is a first total number n.
Here, as shown in
For example, if the length of the PPG sampling signal is 30 minutes and there are 1800 maximum amplitude signal points, then the first total number n=1800, and the peak time data sequence comprises 1800 peak time data.
Step 32, taking the peak time data corresponding to the first index i′ as current peak time data in the peak time data sequence, performing absolute difference calculation on the current peak time data and a peak time data before the current peak time data to generate a scatter point abscissa Xi corresponding to the second index i, and performing absolute difference calculation on the current peak time data and peak time data after the current peak time data to generate a scatter point ordinate Yi corresponding to the second index i; and forming scatter point two-dimensional coordinates XYi by the scatter point abscissa Xi and the scatter point ordinate Yi.
Wherein the scatter point two-dimensional coordinates XYi are (Xi,Yi), the first index i′ is an index number of the peak time data, the value range of the first index i′ is 2 to n−1, the second index i is an index number of the scatter point two-dimensional coordinates XYi, i=′−1, and the value range of the second index i is 1 to n−2.
Here, as shown in
For example, if the first total number n=1800 and the peak time data sequence comprises 1800 peak time data, then,
when i′=2, the second peak time data is used as the current peak time data, the first peak time data is used as the previous peak time data, the third peak time data is used as the next peak time data, the generated the ith (here i=i′−1=2−1=1) scatter point two-dimensional coordinates XY1 is (X1,Y1), where the scatter point abscissa X1=|second peak time data—first peak time data|, and the scatter point ordinate Y1=|third peak time data—second peak time data|, here ∥ is an absolute value operator.
When i′=3, the third peak time data is used as the current peak time data, the second peak time data is used as the previous peak time data, the fourth peak time data is used as the next peak time data, the generated the ith (here i=i′−1=3−1=2) scatter point two-dimensional coordinates XY2 is (X2,Y2), where the scatter point abscissa X2=|third peak time data-second peak time data|, and the scatter point ordinate Y2=|fourth peak time data-third peak time data|.
And so on,
when i′=n−1=1800−1=1799, the 1799th peak time data is used as the current peak time data, the 1798th peak time data is used as the previous peak time data, the 1800th peak time data is used as the next peak time data, the generated ith (here i=i′−1=1799−1=1798) scatter point two-dimensional coordinates XY1798 is (X1798,Y1798), where the scatter point abscissa X1798=|1799th peak time data-1798th peak time &tat and the scatter point ordinate Y1798=|1800th peak time data-1799th peak time data|.
Step 33, forming the scatter point two-dimensional coordinate sequence by n−2 scatter point two-dimensional coordinates XYi.
Wherein the scatter point two-dimensional coordinate sequence is (XY1, . . . XYi, . . . XYn−2).
Here, the total number of scatter points obtained by the PPG sampling signal is n−2.
For example, if the first total number n=1800 and the peak time data sequence comprises 1800 peak time data, then the scatter point two-dimensional coordinate sequence is (XY1, . . . XYi, . . . XY1798).
Step 34, only keeping one of the plurality of identical scatter point two-dimensional coordinates XYi in the scatter point two-dimensional coordinate sequence, deleting the scatter point two-dimensional coordinate XYi of which the scatter point abscissa Xi exceeds the maximum value of inter-beat interval, and deleting the scatter point two-dimensional coordinate XYi of which the scatter point ordinate Yi exceeds the maximum value of the inter-beat interval.
Here, the maximum value of the inter-beat interval is stored in the local storage medium of the equipment.
Here, clear the redundant scatter points first, which are scatter points with the same scatter point two-dimensional coordinates XYi. The clearing method is to keep only one of them and delete the other redundant scatter point two-dimensional coordinates XYi. Then, clear the error scatter points that are too discrete. The embodiment of the present disclosure uses the maximum value of the inter-beat interval to identify the error scatter points. Because both the abscissa and ordinate of the scatter point two-dimensional coordinates XYi are actually the inter-beat intervals, the scatter point two-dimensional coordinates XYi of which the abscissa and ordinate exceed the maximum value of the inter-beat interval are all recorded as the error scatter point and deleted from the scatter point two-dimensional coordinate sequence.
For example, the first total number n=7, the peak time data sequence comprises 7 peak time data, and the scatter point two-dimensional coordinate sequence is (XY1,XY2,XY3,XY4,XY5), wherein XY1 is (1,1), XY2 is (3,1), XY3 is (2,1), XY4 is (1,3), XY5 is (1,1), and the maximum value of the inter-beat interval is 2 seconds. First, delete one of XY1 and XY5 with the same coordinates (assuming XY5 is deleted), then delete the XY2 of which the abscissa exceeds the maximum value of the inter-beat interval and the XY4 of which the ordinate exceeds the maximum value of the inter-beat interval, and finally, after redundancy removal and error removal, the scatter point two-dimensional coordinate sequence is (XY1,XY3), which means that the total number of scatter points at this time have been changed from the previous five to two.
Step 4, performing scatter diagram resolution confirmation processing according to the sampling frequency and a preset maximum value of an inter-beat interval to generate a scatter diagram resolution.
Specifically comprising the following steps: step 41, performing maximum pixel number calculation according to the sampling frequency and the maximum value of the inter-beat interval to generate a maximum number of pixels a, a=sampling frequency * maximum value of inter-beat interval.
Here, if the scatter diagram two-dimensional tensor to be obtained later is regarded as a pixel graph, the tensor shape of the pixel graph needs to be set by calculating a resolution, the resolution is the number of horizontal pixels x the number of vertical pixels, wherein the number of horizontal pixels should be a maximum number of pixels in a horizontal axis direction and the number of vertical pixels should be a maximum number of pixels in a vertical axis direction. It is also known that a maximum boundary value between a horizontal axis and a vertical axis is the maximum value of the inter-beat interval, then after the minimum accuracy in the horizontal axis direction is obtained, the maximum number of pixels in the horizontal axis direction=the maximum boundary value/the minimum accuracy in the horizontal axis direction, and similarly, the maximum number of pixels in the vertical axis direction=the maximum boundary value/the minimum accuracy in the vertical axis direction. Because the interval between two signal points on the PPG sampling signal should be 1/sampling frequency (second), and both the minimum value and minimum accuracy of the inter-beat interval should be 1/sampling frequency, that is, the minimum accuracy in the horizontal axis direction and the vertical axis direction should be 1/ sampling frequency. Therefore,
Because the maximum number of pixels in the horizontal axis direction is equal to the maximum number of pixels in the vertical axis direction, the calculation of the maximum number a of pixels is only done once in step 41.
For example, if the maximum value of the inter-beat interval is 2 seconds and the sampling frequency is 250 Hz, the maximum number of pixels a=2*250=500.
Step 42, setting the number of horizontal pixels X of the scatter diagram resolution as the maximum number of pixels a, and setting the number of vertical pixels Y of the scatter diagram resolution as the maximum number of pixels a.
Here, the scatter diagram resolution=X*Y=a*a.
As mentioned above, the resolution is the number of horizontal pixels×the number of vertical pixels.
For example, if the maximum value of the inter-beat interval is 2 seconds, the sampling frequency is 250 Hz, and the maximum number of pixels a=500, then the scatter diagram resolution is 500×500.
Step 5, performing scatter diagram initialization processing according to the scatter diagram resolution to generate a scatter diagram two-dimensional tensor.
Specifically comprising: setting a scatter diagram two-dimensional tensor according to the scatter diagram resolution.
Wherein the shape of the scatter diagram two-dimensional tensor is H1×W1, H1 is a two-dimensional parameter of the scatter diagram two-dimensional tensor, and H1=Y=a, W1 is a one-dimensional parameter of the scatter diagram two-dimensional tensor, and W1=X=a, the scatter diagram two-dimensional tensor comprises H1*W1 pixel data DS,Z, a value of the pixel data DS,Z is a preset first pixel value, S is a horizontal subscript of the pixel data DS,Z, the value range of S is 1 to W1, Z is a vertical subscript of the pixel data DS,Z, and the value range of Z is 1 to H1.
Here, the first pixel value is stored in the local storage medium of the equipment, and the default first pixel value is set to 0.
Here, the scatter diagram two-dimensional tensor is used to store the pixel values of all the pixels of the scatter diagram. The scatter diagram resolution is the number of horizontal pixels X×the number of vertical pixels Y(a×a). The shape of the scatter diagram two-dimensional tensor should correspond to the scatter diagram resolution, so the two-dimensional parameter and the one-dimensional parameter of the scatter diagram two-dimensional tensor are H1=Y=a and W1=X=a respectively. The pixel data DS,Z is the data in the scatter diagram two-dimensional tensor, and the specific value is a pixel value of a corresponding pixel point.
For example, if the maximum value of the inter-beat interval is 2 seconds, the sampling frequency is 250 Hz, the maximum number of pixels a=500, the scatter diagram resolution is 500×500, and the first pixel value is 0 by default, then H1=Y=a=500, W1=X=a=500, and the shape of the scatter diagram two-dimensional tensor is 500×500. The data of the scatter diagram two-dimensional tensor are:
Wherein the value range of the horizontal subscript S of the pixel data DS,Z if 1 to 500, and the value range of the vertical subscript Z of the pixel data DS,Z is 1 to 500. The value of each pixel data DS,Z is set to 0 by default. At this time, the scatter diagram two-dimensional tensor can be regarded as a blank scatter diagram.
Step 6, performing scatter point marking processing on the scatter diagram two-dimensional tensor according to the scatter point two-dimensional coordinate sequence.
Specifically comprising: in the scatter diagram two-dimensional tensor, setting the value of the pixel data DS,Z matched with the scatter point two-dimensional coordinates XYi of the scatter point two-dimensional coordinate sequence as a preset second pixel value.
Wherein the horizontal subscript S of the pixel data DS,Z is the same as the scatter point abscissa Xi of the scatter point two-dimensional coordinates XYi, and the vertical subscript Z of the pixel data DS,Z is the same as the scatter point ordinate Y, of the scatter point two-dimensional coordinates XYi.
Here, the second pixel value is stored in the local storage medium of the equipment, and the default value of the second pixel value is set to 1.
Here, in the scatter diagram two-dimensional tensor output in step 6, scatter points obtained in step 3 are marked, and the basis of the marking process is the scatter point two-dimensional coordinates output in step 3.
For example, the first total number n=1800, the peak time data sequence comprises 1800 peak time data, and the scatter point two-dimensional coordinate sequence is (XY1, . . . XYi, . . . XY1798), and there is no redundant or error scatter point. The maximum value of the inter-beat interval is 2 seconds, the sampling frequency is 250 Hz, the shape of the scatter diagram two-dimensional tensor is 500×500, the second pixel value is 1 by default, and the data of the scatter diagram two-dimensional tensor are:
Then, in the scatter diagram two-dimensional tensor, setting the value of the pixel data DS,Z matched with the scatter point two-dimensional coordinates XYi of the scatter point two-dimensional coordinate sequence as a preset second pixel value is to set the value of the pixel data DS,Z of which data subscripts are the same as the scatter point two-dimensional coordinates as the second pixel value (set as 1) in a data area of the above two-dimensional tensor. Suppose that the scatter point two-dimensional coordinates XYi in the scatter point two-dimensional coordinate sequence is specifically (1,1), then, the value of D1,1 is set to 1. suppose that the scatter point two-dimensional coordinates XYi in the scatter point two-dimensional coordinate sequence is specifically (9,8), then, the value of D9,8 is set to 1, and so on, until the values of the pixel data DS,Z corresponding to all the scatter point two-dimensional coordinates XYi in the scatter point two-dimensional coordinate sequence is set. At this point, the scatter diagram two-dimensional tensor can be regarded as a scatter diagram that has finished scatter point marking, as shown in the scatter diagram provided by the first embodiment of the present disclosure in
Step 7, using a convolutional neural network of an artificial intelligence network to perform multilayer convolution pooling calculation on the scatter diagram two-dimensional tensor to generate a four-dimensional output tensor.
Wherein the artificial intelligence network, as shown in
Specifically comprising: step 71, according to a four-dimensional tensor input data format of the convolutional neural network, raising the shape of the scatter diagram two-dimensional tensor from a two-dimensional tensor shape to a four-dimensional tensor shape to generate a four-dimensional input tensor.
Wherein the shape of the four-dimensional input four-dimensional tensor is B1×H2×W2×C1, B1 is a four-dimensional parameter of the four-dimensional input four-dimensional tensor, and B1=1, H2 is a three-dimensional parameter of the four-dimensional input four-dimensional tensor, and H2=H1, W2 is a two-dimensional parameter of the four-dimensional input four-dimensional tensor, and W2=W1, C1 is a one-dimensional parameter of the four-dimensional input four-dimensional tensor, and C1=1.
Here, the shape of the scatter diagram two-dimensional tensor is raised from the two-dimensional tensor shape to the four-dimensional tensor shape, and the process only resets the tensor shape without destroying the actual data order in the tensor.
For example, if the scatter diagram two-dimensional tensor [500,500] is raised from the two-dimensional tensor shape to the four-dimensional tensor shape, B1=1, H2=H1=500, W2=W1=500, C1=1, then the shape of the four-dimensional input tensor is 1×500×500×1, which is expressed as the four-dimensional input tensor [1,500,500,1] here.
Step 72, sending the four-dimensional input tensor into a first convolutional network layer of the convolutional neural network for first-layer convolution pooling calculation to generate a first four-dimensional tensor, then sending the first four-dimensional tensor into a second convolutional network layer of the convolutional neural network for second-layer convolution pooling calculation to generate a second four-dimensional tensor, until finally, sending a penultimate four-dimensional tensor into the last convolutional network layer of the convolutional neural network for last-layer convolution pooling calculation to generate the four-dimensional output tensor.
Wherein the convolutional neural network comprises a plurality of convolutional network layers, the convolutional network layer comprises a convolution layer and a pooling layer, the shape of the four-dimensional output tensor is B2×H3×W3×C2, B2 is a four-dimensional parameter of the four-dimensional output tensor, and B2=B1=1, H3 is a three-dimensional parameter of the four-dimensional output tensor, W3 is a two-dimensional parameter of the four-dimensional output tensor, and C2 is a one-dimensional parameter of the four-dimensional output tensor.
Wherein, the four-dimensional input tensor is sent to the first convolutional network layer of the convolutional neural network for first-layer convolution pooling calculation to generate the first four-dimensional tensor, specifically, the four-dimensional input tensor is sent to the first convolution layer of the first convolutional network layer for the first convolution calculation to generate a first convolution four-dimensional tensor, and then, the first convolution four-dimensional tensor is sent to the first pooling layer of the first convolutional network layer for the first pooling calculation to generate the first four-dimensional tensor.
For example, the convolutional neural network comprises four convolutional network layers, and its network structure is shown in a structural diagram schematic of a convolutional neural network provided by the first embodiment of the present disclosure in
the four-dimensional input tensor is sent into the first convolution layer of the first convolutional network layer of the convolutional neural network for the first convolution calculation to generate a first convolution four-dimensional tensor, and the first convolution four-dimensional tensor is sent into the first pooling layer for the first pooling calculation to generate the first four-dimensional tensor.
The first four-dimensional tensor is sent into a second convolution layer of a second convolutional network layer of the convolutional neural network for the second convolution calculation to generate a second convolution four-dimensional tensor, and the second convolution four-dimensional tensor is sent into a second pooling layer for the second pooling calculation to generate a second four-dimensional tensor.
The second four-dimensional tensor is sent into a third convolution layer of a third convolutional network layer of the convolutional neural network for the third convolution calculation to generate a third convolution four-dimensional tensor, and the third convolution four-dimensional tensor is sent into a third pooling layer for the third pooling calculation to generate a third four-dimensional tensor, and
the third four-dimensional tensor is sent into a fourth convolution layer of a fourth convolutional network layer of the convolutional neural network for the fourth convolution calculation to generate a fourth convolution four-dimensional tensor, and the fourth convolution four-dimensional tensor is sent into a fourth pooling layer for the fourth pooling calculation to finally obtain the four-dimensional input tensor.
As can be seen from the foregoing, in the convolutional neural network, after each convolution layer or pooling layer, the shape of the input data will change, but the four-dimensional tensor form does not change, and the four-dimensional parameter will not change. The change of the three-dimensional and second-dimensional parameters is related to the size of a convolution kernel and the setting of a sliding step of each convolution layer, as well as the size of a pooling window and a sliding step of the pooling layer, and the change of the one-dimensional parameter is related to a selected output space dimension (the number of convolution kernels) in the convolution layer. In practical application, the setting of the number of layers in the network and various parameters of each layer may be constantly revised according to experience and experimental results.
Step 8, using the fully connected neural network of the artificial intelligence network to perform multi-layer full connection calculation on the four-dimensional output tensor to generate a two-dimensional output tensor.
Specifically comprising: step 81, according to a two-dimensional tensor input data format of the fully connected neural network, reducing the shape of the four-dimensional output tensor from a four-dimensional tensor shape to a two-dimensional tensor shape to generate a two-dimensional input tensor.
Wherein the shape of the two-dimensional input tensor is B3×W4, B3 is a two-dimensional parameter of the two-dimensional input tensor, and B3=B2=1, W4 is a one-dimensional parameter of the two-dimensional input tensor, and W4=H3*W3*C2.
Here, the shape of the four-dimensional output tensor is reduced from the four-dimensional tensor shape to the two-dimensional tensor shape, and the process only resets the tensor shape without destroying the actual data order in the tensor.
For example, if the shape of the output four-dimensional tensor of the convolutional neural network is 1×2×20×64, then B3=B2=1, W8=H3*W3*C2=2*20*64=2560, and the shape of the two-dimensional input tensor should be 1×2560, which is expressed here as the two-dimensional input tensor [1,2560].
Step 82, sending the two-dimensional input tensor into a first fully connected network layer of the fully connected neural network for first-layer full connection calculation to generate a first two-dimensional tensor, then sending the first two-dimensional tensor into a second fully connected network layer of the fully connected neural network for second-layer full connection calculation to generate a second two-dimensional tensor, and finally, sending a penultimate two-dimensional tensor into a last fully connected network layer of the fully connected neural network for last-layer full connection calculation to generate the two-dimensional output tensor.
Wherein the fully connected neural network comprises multiple fully connected layers, the shape of the two-dimensional output tensor is B4×W5, B4 is a two-dimensional parameter of the two-dimensional output tensor, and B4=B3=1, W5 is a one-dimensional parameter of the two-dimensional output tensor, and W5=2, and the two-dimensional output tensor comprises two data: classification reasonable weight data and classification unreasonable weight data.
For example, the fully connected neural network comprises four fully connected layers, and its network structure is shown in
the two-dimensional input tensor is sent into the first fully connected layer of the fully connected neural network for first-layer full connection calculation to generate the first two-dimensional tensor.
The first two-dimensional tensor is sent into the second fully connected layer of the fully connected neural network for second-layer full connection calculation to generate the second two-dimensional tensor.
The second two-dimensional tensor is sent into a third fully connected layer of the fully connected neural network for third-layer full connection calculation to generate a third two-dimensional tensor.
The third two-dimensional tensor is sent into a fourth fully connected layer of the fully connected neural network for second-layer full connection calculation to finally obtain the two-dimensional output tensor.
Here, B4=B3=1, the number of nodes in the last fully connected layer is 2(W5=2), and the shape of the corresponding final two-dimensional output tensor is specifically 1×2, which is expressed as two-dimensional output tensor [1,2], and the data of the two-dimensional output tensor [1,2] is (classification reasonable weight data, classification unreasonable weight data).
Step 9, using the normalization processing layer of the artificial intelligence network to perform normalization index calculation on the two-dimensional output tensor to generate a normalization two-dimensional tensor.
Specifically comprising: step 91, sending the two-dimensional output tensor into a normalization processing layer for normalization index calculation of the classification reasonable weight data and the classification unreasonable weight data to generate a classification reasonable probability and a classification unreasonable probability.
Wherein the sum of the classification reasonable probability and the classification unreasonable probability is 1.
Here, the reason why the classification reasonable weight data and the classification unreasonable weight data are subjected to normalization index processing is that through a clear ratio relationship, on the one hand, it can be clear at a glance, and on the other hand, more probability data can be collected for trend analysis. Because the normalization is based on two data, the sum of the two probabilities should be 1.
Step 92, forming a normalization two-dimensional tensor by the classification reasonable probability and the classification unreasonable probability;
Wherein the shape of the normalization two-dimensional tensor is B5×W6, B5 is a two-dimensional parameter of the normalization two-dimensional tensor, and B5=B4=1, W6 is a one-dimensional parameter of the normalization two-dimensional tensor, and W6=W5=2, and the normalization two-dimensional tensor comprises two data: classification reasonable probability and classification unreasonable probability.
For example, if the shape of the two-dimensional output tensor is 1×2, then B5=B4=1, W6=W5=2, the shape of the normalization two-dimensional tensor is also 1×2, which is expressed as two-dimensional output tensor [1,2], and the data of the two-dimensional output tensor [1,2] is(classification reasonable probability, classification unreasonable probability).
Step 10, performing classification confirmation according to the normalization two-dimensional tensor to generate confirmation data.
Wherein the confirmation data comprise classification reasonable information and classification unreasonable information.
Specifically comprising: identifying whether the classification reasonable probability is higher than the classification unreasonable probability, and when the classification reasonable probability is higher than the classification unreasonable probability, taking the classification reasonable information as confirmation data, and when the classification reasonable probability is lower than the classification unreasonable probability, taking the classification unreasonable information as confirmation data.
Here, the confirmation data including the classification reasonable information and the classification unreasonable information are information stored in the local storage medium of the equipment.
Here, when the classification reasonable probability is higher than the classification unreasonable probability, it means that the probability of the artificial intelligence network identifying the current scatter diagram as a scatter diagram of a certain type is higher than the probability that it is not of the type. When the classification reasonable probability is lower than the classification unreasonable probability, it means that the probability of the artificial intelligence network identifying the current scatter diagram as a scatter diagram not of a certain type is higher than the probability that it is of the type.
For example, the artificial intelligence network is used to identify an atrial fibrillation scatter diagram, and the data of the two-dimensional output tensor [1,2] is (90%, 10%), where the classification reasonable probability=90%, the classification unreasonable probability=10%, the classification reasonable information is specifically “atrial fibrillation state,” and the classification unreasonable information is specifically “non-atrial fibrillation state,” then in the embodiment of the present disclosure, the confirmation data is set to classification reasonable information (“atrial fibrillation state”). After separate confirmation, the equipment gets the confirmation data of “atrial fibrillation state,” and will immediately start a related early warning process.
The scatter diagram classification device for a PPG signal provided by the embodiment of the present disclosure can perform the method steps in the above method embodiment, and its implementation principles and technical effects are similar, which will not be repeated here.
It should be understood that the division of different modules of the above device is based on logical functions, and in actual implementation, all or part of the modules can be integrated into a physical entity, or they can be physically separated. These modules can all be implemented in the form of software that can be called by processing elements, or all of them can be implemented in the form of hardware, or some modules are implemented in the form of software that can be called by processing elements, and some modules are implemented in the form of hardware. For example, the acquisition module may be a separate processing element, or may be integrated into a certain chip of the above-mentioned device, or it may be stored in a memory of the above-mentioned device in the form of program code, and called by a certain processing element of the above-mentioned device to implement the functions of the above-mentioned determination module. Other modules are implemented similarly. In addition, all or part of these modules may be integrated or implemented separately. The processing element described here may be an integrated circuit with signal processing capability. In the implementation process, each step of the above method or each module may be realized by an integrated logic circuit of hardware in the processor element or instructions in the form of software.
For example, the above modules may be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASIC), one or more digital signal processors (DSP), one or more field programmable gate arrays (FPGA), etc. For another example, when one of the above modules is implemented in the form of a program code that can be called by a processing element, the processing element may be a general purpose processor, such as a central processing unit (CPU) or other processors that can call the program code. For example, these modules can be integrated and implemented in the form of system-on-a-chip (SOC).
In the above embodiments, the functional units can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, the functional units can be implemented in whole or in part by computer program products. The computer program product comprises one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function according to the embodiment of the disclosure is generated in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in the computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one web site, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, Bluetooth and microwave) methods. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as server and data center that contains one or more available media integrations. The available medium may be magnetic medium (e.g., floppy disk, hard disk, magnetic tape), optical medium (e.g., DVD), or semiconductor medium (e.g., solid state disk (SSD)).
The system bus mentioned in
The above processor may be a general purpose processor, including a CPU, a network processor (NP), a graphics processing unit (GPU), etc., and may also be a DSP, an ASIC, an FPGA or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.
It should be noted that the embodiment of the present disclosure also provides a computer-readable storage medium, in which instructions are stored, which, when run on a computer, cause the computer to execute the method and processing procedures provided in the above embodiments.
An embodiment of the disclosure also provides a chip for running instructions, and the chip is configured to execute the method and processing procedures provided in the above embodiments.
An embodiment of the disclosure also provides a program product, which comprises a computer program stored in a storage medium. At least one processor may read the computer program from the storage medium, and the at least one processor executes the method and processing procedures provided in the above embodiments.
The embodiment of the disclosure provides a scatter diagram classification method and device for a PPG signal, electronic equipment, a computer program product and a computer-readable storage medium. By extracting inter-beat interval data from PPG signals to generate a scatter diagram, and then introducing the scatter diagram into an artificial intelligence network for confirming the type of the scatter diagram for type confirmation, therefore the scatter diagram generation difficulty is reduced, and the application scenarios of PPG in the field of health monitoring are enriched.
Professionals should further realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software or a combination of the two. In order to clearly explain the interchangeability of hardware and software, the components and steps of each example have been generally described according to functions in the above description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical scheme. Professionals can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the disclosure.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be implemented in hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable magnetic disk, CD-ROM, or any other form of storage medium known in the technical field.
The above-mentioned specific embodiments further explain the purpose, technical scheme and beneficial effects of the invention in detail. It should be understood that the above are only specific embodiments of the invention and are not used to limit the scope of protection of the disclosure. Any modification, equivalent substitution, improvement, etc., made within the spirit and principles of the disclosure should be included in the scope of protection of the disclosure.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202011086771.9 | Oct 2020 | CN | national |
This application is a national phase entry under 35 U.S.C. § 371 of International Patent Application PCT/CN2021/088021, filed Apr. 19, 2021, designating the United States of America and published as International Patent Publication WO 2022/077888 A1 on Apr. 21, 2022, which claims the benefit under Article 8 of the Patent Cooperation Treaty to Chinese Patent Application Serial No. 202011086771.9, filed in the National Intellectual Property Administration, PRC, on Oct. 12, 2020, for “Scatter Diagram Classification Method and Device for Photoplethysmography (PPG) Signal.”
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2021/088021 | 4/19/2021 | WO |
