Patent Application
20230293144
The present invention relates to an ultrasound diagnostic apparatus, and a control method of the ultrasound diagnostic apparatus which have a function of determining a type (an artery or a vein) of a blood vessel.
For example, in a case of puncturing a blood vessel of a subject, in order to determine a puncture position and the thickness of a puncture needle, an ultrasound image is observed by using an ultrasound diagnostic apparatus to check the type, size, position, depth, and traveling of the blood vessel, and the presence or absence of an obstacle such as nerves. In particular, since mistakes in puncturing arteries and veins directly lead to serious medical accidents, it is important to determine the type of the blood vessel, that is, whether the blood vessel is an artery or a vein.
Here, there are JP2008-545502A and JP2018-202147A as the documents in the related art that are references for the present invention.
JP2008-545502A discloses that arteries and veins are distinguished by applying a color Doppler technique to detect the direction and pulsation of the blood flow in the arteries and the veins. JP2008-545502A discloses that arteries and veins are distinguished by using the fact that the veins collapse in a case where a mechanical pressure is applied to a tissue with blood vessels, or by measuring and analyzing the absorption of two wavelengths on the basis of the oxygen content in the blood.
JP2018-202147A discloses that arteries and veins are decided by analyzing image data using anatomical prior information indicating typical positions of the arteries and the veins.
As in JP2018-202147A, in a case where the type of the blood vessel is determined only on the basis of the image data, erroneous determination may be performed. As described above, since erroneous determination of the type of the blood vessel directly leads to medical accidents, it is desirable to avoid that the type of the blood vessel is determined only on the basis of the image data.
On the other hand, as in JP2008-545502A, in a case where the type of the blood vessel is determined on the basis of data other than the image data, additional work is always required to determine the type of the blood vessel, which causes the deterioration of the usability.
An object of the present invention is to provide an ultrasound diagnostic apparatus and a control method of the ultrasound diagnostic apparatus which can prevent the type of the blood vessel from being erroneously determined, and determine the type of the blood vessel without the deterioration of the usability.
In order to achieve the object, an aspect of the present invention provides an ultrasound diagnostic apparatus including an ultrasound probe; an image generation unit that generates an ultrasound image from a reception signal obtained by performing transmission and reception of an ultrasound beam with respect to a subject using the ultrasound probe; a probability calculation unit that calculates each of a probability that a blood vessel in the ultrasound image is an artery and a probability that the blood vessel is a vein by analyzing the ultrasound image; a blood vessel determination unit that determines whether the blood vessel is the artery or the vein or whether or not confirmation processing for confirming whether the blood vessel is the artery or the vein is required on the basis of the probability that the blood vessel is the artery and the probability that the blood vessel is the vein; and a blood vessel confirmation unit that executes the confirmation processing in a case where it is determined that the confirmation processing is required, and confirms whether the blood vessel is the artery or the vein on the basis of an execution result of the confirmation processing.
Here, it is preferable that the probability calculation unit detects a blood vessel region with a predetermined shape, including a blood vessel in the ultrasound image, in a case of calculating a probability of being the artery and a probability of being the vein.
Further, it is preferable that the blood vessel region is an artery region including the artery or a vein region including the vein, and the probability calculation unit detects the artery region including the blood vessel in the ultrasound image, calculates a probability that the blood vessel in the artery region is the artery, detects the vein region including the same blood vessel, and calculates a probability that the same blood vessel in the vein region is the vein.
It is preferable that the probability calculation unit detects a single blood vessel region with the predetermined shape, including a blood vessel in the ultrasound image, and further calculates a probability that the blood vessel in the single blood vessel region is the artery and a probability that the blood vessel in the single blood vessel region is the vein.
It is preferable that, in a first case in which a probability of being one of the artery and the vein is equal to or greater than a first threshold value and a probability of being the other of the artery and the vein is less than a second threshold value smaller than the first threshold value, the blood vessel determination unit determines that the blood vessel is one of the artery and the vein, and in a second case other than the first case, the blood vessel determination unit determines that the confirmation processing is required.
It is preferable that the probability calculation unit calculates, for each pixel of the ultrasound image, each of a probability that the pixel is a pixel of the artery, a probability that the pixel is a pixel of the vein, and a probability that the pixel is a pixel of background other than the blood vessel.
It is preferable that the blood vessel determination unit decides, for each pixel of the ultrasound image, whether the pixel is the pixel of the artery, the pixel is the pixel of the vein, or the pixel is the pixel of the background on the basis of the probability that the pixel is the pixel of the artery, the probability that the pixel is the pixel of the vein, and the probability that the pixel is the pixel of the background, decides a blood vessel region including the blood vessel on the basis of the pixel of the artery and the pixel of the vein, determines that the blood vessel is one of the artery and the vein in a case where a ratio of one of an area of the pixel of the artery and an area of the pixel of the vein to an area of the blood vessel in the blood vessel region is equal to or greater than a third threshold value, and determines that the confirmation processing is required in a case where the ratio is less than the third threshold value.
It is preferable that the blood vessel confirmation unit prompts a user to press an examination location of the subject with the ultrasound probe, calculates an aspect ratio of a cross section of the blood vessel on the basis of an ultrasound image generated while the examination location is pressed with the ultrasound probe, and confirms whether the blood vessel is the artery or the vein on the basis of the aspect ratio of the cross section of the blood vessel.
It is preferable that the ultrasound probe has a pressure sensor, and the blood vessel confirmation unit determines whether or not the examination location is pressed on the basis of a pressure detected by the pressure sensor.
It is preferable that the blood vessel confirmation unit detects motion in the examination location by analyzing the ultrasound images of a plurality of frames, and determines whether or not the examination location is pressed on the basis of the motion in the examination location.
It is preferable that the blood vessel confirmation unit determines presence or absence of a change in the aspect ratio of the cross section of the blood vessel only for a predetermined period of time after prompting the user to press the examination location with the ultrasound probe, and determines whether or not the examination location is pressed on the basis of the presence or absence of the change in the aspect ratio of the blood vessel in the predetermined period of time.
It is preferable that the blood vessel confirmation unit prompts a user to incline the ultrasound probe at an examination location of the subject, and confirms whether the blood vessel is the artery or the vein on the basis of a direction of a blood flow in an ultrasound image generated while the ultrasound probe is inclined at the examination location in a color Doppler mode.
It is preferable that the blood vessel confirmation unit searches for an ultrasound image of a frame in which a probability that the same blood vessel is the artery or a probability that the same blood vessel is the vein is equal to or greater than a fourth threshold value among ultrasound images of past frames, and confirms whether the blood vessel is the artery or the vein on the basis of the probability that the same blood vessel in the ultrasound image of the searched frame is the artery or the probability that the same blood vessel is the vein.
It is preferable that the blood vessel confirmation unit detects the blood vessel in ultrasound images of adjacent preceding and succeeding frames, and determines that the blood vessels are the same in a case where an overlapping ratio of the blood vessels in the ultrasound images of the preceding and succeeding frames is equal to or greater than the fourth threshold value.
It is preferable that the ultrasound diagnostic apparatus further includes a monitor; and a display control unit that superimposes a graphic including the blood vessel on the ultrasound image to be displayed on the monitor on the basis of at least one of a determination result on whether the blood vessel is the artery or the vein or a confirmation result on whether the blood vessel is the artery or the vein.
It is preferable that the display control unit displays each of the probability that the blood vessel is the artery and the probability that the blood vessel is the vein on the monitor in addition to the graphic.
It is preferable that the display control unit displays the same graphic for the same blood vessel on the monitor once whether the blood vessel is the artery or the vein has been decided.
It is preferable that the display control unit displays characters, blood vessel, on the monitor in addition to the graphic in a case where deciding whether the blood vessel is the artery or the vein is not possible.
Another aspect of the present invention provides a control method of an ultrasound diagnostic apparatus, the control method including generating an ultrasound image from a reception signal obtained by performing transmission and reception of an ultrasound beam with respect to a subject using an ultrasound probe; calculating a probability that a blood vessel in the ultrasound image is an artery and a probability that the blood vessel is a vein by analyzing the ultrasound image; determining whether the blood vessel is the artery or the vein or whether or not confirmation processing for confirming whether the blood vessel is the artery or the vein is required on the basis of the probability that the blood vessel is the artery and the probability that the blood vessel is the vein; and executing the confirmation processing in a case where it is determined that the confirmation processing is required, and confirming whether the blood vessel is the artery or the vein on the basis of an execution result of the confirmation processing.
In the present invention, the type of the blood vessel is determined on the basis of the probability that the blood vessel is an artery and the probability that the blood vessel is a vein as described above, and in a case where the type of the blood vessel cannot be determined, the confirmation processing is executed to confirm whether the blood vessel is an artery or a vein. Therefore, it is possible to prevent the type of the blood vessel from being erroneously determined, and to greatly improve the determination accuracy of the type of the blood vessel. Further, in the present invention, since the confirmation processing is executed only in a case where the type of the blood vessel cannot be determined, it is possible to greatly suppress the deterioration of the usability.
Hereinafter, an ultrasound diagnostic apparatus and a control method of the ultrasound diagnostic apparatus according to the present invention will be described in detail on the basis of preferred embodiments illustrated in the accompanying drawings.
The ultrasound probe 1 scans an examination location of a subject using an ultrasound beam, and outputs a sound ray signal corresponding to an ultrasound image of the examination location. As illustrated in
The transducer array 11 has a plurality of ultrasonic transducers arranged in a one-dimensional or two-dimensional manner. According to a drive signal supplied from the transmission and reception circuit 14, each of the transducers transmits an ultrasonic wave and receives a reflected wave from the subject to output an analog reception signal.
For example, each transducer is formed by using an element in which electrodes are formed at both ends of a piezoelectric body consisting of piezoelectric ceramic represented by lead zirconate titanate (PZT), a polymer piezoelectric element represented by poly vinylidene di fluoride (PVDF), piezoelectric single crystal represented by lead magnesium niobate-lead titanate (PMN-PT), or the like.
The transmission and reception circuit 14 causes the transducer array 11 to transmit the ultrasonic wave, and performs reception focusing processing on the reception signal output from the transducer array 11 that has received the ultrasound echo to generate a sound ray signal, under the control of the apparatus control unit 36. As illustrated in
The pulser 51 includes, for example, a plurality of pulse generators, and the pulser 51 adjusts the amount of delay of each drive signal so that ultrasonic waves transmitted from the plurality of transducers of the transducer array 11 form an ultrasound beam on the basis of a transmission delay pattern selected by the apparatus control unit 36, and supplies the obtained signals to the plurality of transducers. Thus, in a case where a pulsed or continuous-wave voltage is applied to the electrodes of the transducers of the transducer array 11, the piezoelectric body expands and contracts to generate pulsed or continuous-wave ultrasonic waves from each transducer. From the combined wave of these ultrasonic waves, an ultrasound beam is formed.
The transmitted ultrasound beam is reflected by a target, for example, a site of the subject, and propagates toward the transducer array 11 of the ultrasound probe 1. Each transducer constituting the transducer array 11 expands and contracts by receiving the ultrasound echo propagating toward the transducer array 11 in this manner, to generate the reception signal that is an electric signal, and outputs the reception signal to the amplification unit 52.
The amplification unit 52 amplifies the signals input from each transducer constituting the transducer array 11, and transmits the amplified signals to the AD conversion unit 53. The AD conversion unit 53 converts the analog signal transmitted from the amplification unit 52 into digital reception data, and outputs the reception data to the beam former 54.
The beam former 54 performs so-called reception focusing processing in which addition is performed by giving delays to respective pieces of the reception data converted by the AD conversion unit 53 according to a sound speed distribution or a sound speed set on the basis of a reception delay pattern selected by the apparatus control unit 36. Through the reception focusing processing, a sound ray signal in which each piece of the reception data converted by the AD conversion unit 53 is phased and added and the focus of the ultrasound echo is narrowed is generated.
Next, the apparatus main body 3 generates an ultrasound image of the examination location of the subject on the basis of the sound ray signal generated by the ultrasound probe 1, and displays the ultrasound image of the examination location of the subject. As illustrated in
The image generation unit 31 is connected to the transmission and reception circuit 14, and the display control unit 33 and the monitor 34 are sequentially connected in series to the image generation unit 31. Each of the image memory 32 and the blood vessel processing unit 35 is connected to the image generation unit 31, and the display control unit 33 is connected to the image memory 32 and the blood vessel processing unit 35. The apparatus control unit 36 is connected to the transmission and reception circuit 14, the image generation unit 31, the display control unit 33, the image memory 32, and the blood vessel processing unit 35, and the input device 37 is connected to the apparatus control unit 36.
The image generation unit 31 generates the ultrasound image (ultrasound image signal) of the examination location of the subject, from the reception signal obtained by performing transmission and reception of the ultrasound beams with respect to the examination location of the subject using the ultrasound probe 1 (more precisely, transducer array 11), in other words, from the sound ray signal generated from the reception signal by the transmission and reception circuit 14, under the control of the apparatus control unit 36. As illustrated in
The signal processing unit 16 generates image information data corresponding to the ultrasound image on the basis of the sound ray signal generated by the transmission and reception circuit 14. More specifically, the signal processing unit 16 generates the image information data representing tomographic image information regarding tissues inside the subject, by performing envelope detection processing after signal processing, for example, correcting the attenuation of the sound ray signal generated by the beam former 54 of the transmission and reception circuit 14, which is caused by the propagation distance according to the depth of the reflection position of the ultrasonic wave.
The DSC 18 raster-converts the image information data generated by the signal processing unit 16 into an image signal according to a normal television signal scanning method.
The image processing unit 17 performs various kinds of image processing such as brightness correction, gradation correction, sharpness correction, image size correction, refresh rate correction, scanning frequency correction, and color correction according to a display format of the monitor 34, on the image signal input from the DSC 18 to generate the ultrasound image (ultrasound image signal), and then outputs the ultrasound image on which the image processing has been performed, to the image memory 32, the blood vessel processing unit 35, and the display control unit 33.
The image memory 32 is a memory that stores ultrasound images (ultrasound image signal) of the series of a plurality of frames, which are generated for each examination location by the image generation unit 31, under the control of the apparatus control unit 36. Here, as the image memory 32, recording media such as a flash memory, a hard disk drive (HDD), a solid state drive (SSD), a flexible disc (FD), a magneto-optical disc (MO disc), a magnetic tape (MT), a random access memory (RAM), a compact disc (CD), a digital versatile disc (DVD), a secure digital card (SD card), and a universal serial bus memory (USB memory), an external server, or the like can be used.
The blood vessel processing unit 35 performs various kinds of processing for determining the type of the blood vessel in the ultrasound image of the examination location of the subject, that is, whether the blood vessel is an artery or a vein, under the control of the apparatus control unit 36. As illustrated in
The probability calculation unit 61 calculates each of a probability that the blood vessel in the ultrasound image is an artery and a probability that the blood vessel in the ultrasound image is a vein by analyzing the ultrasound image of the examination location of the subject.
Although not particularly limited, the probability calculation unit 61 can calculate the probability that the blood vessel is an artery and the probability that the blood vessel is a vein by analyzing the ultrasound image using at least one of a machine learning model trained using a machine learning technique such as deep learning, template matching, and an image analysis technique using feature amounts such as Adaptive Boosting (Adaboost), support vector machines (SVM) or scale-invariant feature transform (SIFT).
The blood vessel determination unit 62 determines whether the blood vessel is an artery or a vein, or whether or not additional confirmation processing for confirming whether the blood vessel is an artery or a vein is required, on the basis of the probability that the blood vessel is an artery and the probability that the blood vessel is a vein. In other words, the blood vessel determination unit 62 determines whether the type of the blood vessel can be decided or cannot be decided, only on the basis of the probability that the blood vessel is an artery or the probability that the blood vessel is a vein.
The confirmation processing will be specifically described later.
In a case where it is determined that the confirmation processing is required by the blood vessel determination unit 62, the blood vessel confirmation unit 63 executes the confirmation processing, and confirms that the blood vessel in the ultrasound image is an artery or a vein on the basis of an execution result of the confirmation processing.
The display control unit 33 displays various kinds of information on the monitor 34 under the control of the apparatus control unit 36. For example, the display control unit 33 performs predetermined processing on the ultrasound image held in the image memory 32, and displays the processed ultrasound image on the monitor 34. Further, the display control unit 33 displays a determination result on whether the blood vessel is an artery or a vein, a confirmation result on whether the blood vessel is an artery or a vein, and the like on the monitor 34.
The monitor 34 displays various kinds of information described above under the control of the display control unit 33. Examples of the monitor 34 include a display device such as a liquid crystal display (LCD), and an organic electroluminescence (EL) display.
The input device 37 receives various instructions input from the user (examiner) of the ultrasound diagnostic apparatus. Although not particularly limited, the input device 37 includes various buttons, and a touch panel or the like through which various instructions are input by the user performing a touch operation.
The apparatus control unit 36 controls the ultrasound probe 1 and each unit of the apparatus main body 3 on the basis of a program stored in advance and an instruction or the like of the user input from the input device 37.
The image generation unit 31, the blood vessel processing unit 35, the display control unit 33, and the apparatus control unit 36 constitute a terminal-side processor 39.
Next, the operation of the ultrasound diagnostic apparatus in a case where the ultrasound image is captured will be described with reference to the flowchart of
First, in a state where the ultrasound probe 1 is in contact with the examination location of the subject, under the control of the apparatus control unit 36, the transmission of the ultrasonic waves is started by the transmission and reception circuit 14, and the sound ray signal is generated (Step S1).
That is, the ultrasound beams are transmitted to the examination location of the subject from a plurality of transducers of the transducer array 11 according to the drive signals from the pulser 51.
Ultrasound echoes from the examination location based on the ultrasound beams transmitted from the pulser 51 are received by each transducer of the transducer array 11, and the reception signal as an analog signal is output from each transducer of the transducer array 11, which has received the ultrasound echo.
The reception signal output from each transducer of the transducer array 11 is amplified by the amplification unit 52, and is subjected to AD conversion by the AD conversion unit 53, and thereby the reception data is acquired.
By performing the reception focusing processing on the reception data by the beam former 54, the sound ray signal is generated.
Next, under the control of the apparatus control unit 36, the ultrasound image (ultrasound image signal) of the examination location of the subject is generated by the image generation unit 31 on the basis of the sound ray signal generated by the beam former 54 of the transmission and reception circuit 14 (Step S2).
That is, the sound ray signal generated by the beam former 54 is subjected to various kinds of signal processing by the signal processing unit 16, and the image information data representing tomographic image information regarding tissues inside the subject is generated.
The image information data generated by the signal processing unit 16 is raster-converted by the DSC 18, and is further subjected to various kinds of image processing by the image processing unit 17, and thus the ultrasound image (ultrasound image signal) is generated.
The ultrasound image generated by the image processing unit 17 is held in the image memory 32.
Next, under the control of the apparatus control unit 36, predetermined processing is performed on the ultrasound image held in the image memory 32 by the display control unit 33, and the processed ultrasound image is displayed on the monitor 34 (Step S3).
Next, the operation of the ultrasound diagnostic apparatus in a case of determining the type of the blood vessel in the ultrasound image will be described. First, with reference to the flowchart of
First, by the probability calculation unit 61, the blood vessel region in the ultrasound image is detected, and the probability that the blood vessel in the blood vessel region is an artery and the probability that the blood vessel in the blood vessel region is a vein are calculated (Step S11).
The blood vessel region is one of an artery region including an artery or a vein region including a vein.
For example, the probability calculation unit 61 detects the artery region including the blood vessel in the ultrasound image, and calculates the probability that the blood vessel in the artery region is an artery, by using the machine learning model. Further, the probability calculation unit 61 detects the vein region including the same blood vessel, and calculates the probability that the blood vessel in the vein region is a vein. That is, the probability calculation unit 61 performs the calculation of the probability that the blood vessel is an artery and the calculation of the probability that the blood vessel is a vein, for the same single blood vessel in two steps.
Here, the machine learning model of the artery is a trained model in which learning ultrasound images and ground truth data of the artery region including the blood vessel in the learning ultrasound image and the probability that the blood vessel in the artery region is an artery are used as teacher data, and which has been trained for a relationship between the learning ultrasound image and the ground truth data of the artery region and the probability that the blood vessel in the artery region is an artery in the learning ultrasound image, for a plurality of pieces of the teacher data.
In this case, in the machine learning model of the artery, the ultrasound image of the examination location of the subject at the time of examination is input, and an estimation result of estimating the artery region in the ultrasound image and the probability that the blood vessel in the artery region is an artery is output.
Then, by the probability calculation unit 61, the artery region in the ultrasound image is detected, and the probability that the blood vessel in the artery region is an artery is calculated on the basis of the estimation result by the machine learning model of the artery.
Further, the machine learning model of the vein is a trained model in which learning ultrasound images and ground truth data of the vein region in the learning ultrasound image and the probability of being a vein are used as teacher data, and which has been trained for a relationship between the learning ultrasound image and the ground truth data of the vein region and the probability of being a vein in the learning ultrasound image, for a plurality of pieces of the teacher data.
In this case, in the machine learning model of the vein, the ultrasound image of the examination location of the subject at the time of examination is input, and an estimation result of estimating the vein region in the ultrasound image and the probability of being a vein is output.
Then, by the probability calculation unit 61, the vein region in the ultrasound image is detected, and the probability that the blood vessel in the vein region is a vein is calculated on the basis of the estimation result by the machine learning model of the vein.
Similarly, the probability calculation unit 61 may detect a single blood vessel region with a predetermined shape, including the blood vessel in the ultrasound image, and calculate the probability that the blood vessel in the single blood vessel region is an artery and the probability that the blood vessel in the single blood vessel region is a vein, by using the machine learning model. The single blood vessel region means a region including a single blood vessel without distinguishing the artery region or the vein region. That is, the probability calculation unit 61 performs the calculation of the probability that the blood vessel is an artery and the probability that the blood vessel is a vein at once.
Here, the machine learning model is a trained model in which learning ultrasound images and ground truth data of the single blood vessel region with a predetermined shape, including the blood vessel in the learning ultrasound image, the probability that the blood vessel in the single blood vessel region is an artery, and the probability that the blood vessel in the single blood vessel region is a vein are used as teacher data, and which has been trained for a relationship between the learning ultrasound image and the ground truth data of the single blood vessel region in the learning ultrasound image, the probability that the blood vessel in the single blood vessel region is an artery, and the probability that the blood vessel in the single blood vessel region is a vein, for a plurality of pieces of the teacher data.
In this case, in the machine learning model, the ultrasound image of the examination location of the subject at the time of examination is input, and an estimation result of estimating the single blood vessel region in the ultrasound image, the probability that the blood vessel in the single blood vessel region is an artery, and the probability that the blood vessel in the single blood vessel region is a vein is output.
Then, by the probability calculation unit 61, the single blood vessel region in the ultrasound image is detected, and the probability that the blood vessel in the single blood vessel region is an artery and the probability that the blood vessel in the single blood vessel region is a vein are calculated on the basis of the estimation result by the machine learning model.
Subsequently, by the blood vessel determination unit 62, whether the blood vessel is an artery or a vein, or whether or not the confirmation processing is required is determined on the basis of the probability that the blood vessel is an artery and the probability that the blood vessel is a vein (Step S12).
Here, in a first case in which the probability of being one of an artery and a vein is equal to or greater than a first threshold value, and the probability of being the other of an artery and a vein is less than a second threshold value smaller than the first threshold value (Yes in Step S12), it is determined that the blood vessel is one of an artery or a vein by the blood vessel determination unit 62 (Step S13).
In a case where it is determined that the blood vessel is one of an artery or a vein, the graphic including the blood vessel is superimposed on the ultrasound image to be displayed on the monitor 34 on the basis of the determination result by the display control unit 33 (Step S14). For example, as illustrated in
In the present invention, displaying the probability that the blood vessel is an artery and the probability that the blood vessel is a vein on the monitor 34 is not essential. That is, the probability that the blood vessel is an artery and the probability that the blood vessel is a vein may be displayed or may not be displayed on the monitor 34.
In the example illustrated in
As illustrated in
In
On the other hand, in a second case other than the first case described above (No in Step S12), that is, in a case other than the case in which the probability that the blood vessel is one of an artery and a vein is equal to or greater than the first threshold value, and the probability that the blood vessel is the other of an artery and a vein is less than the second threshold value smaller than the first threshold value, it is determined that the additional confirmation processing is required by the blood vessel determination unit 62 (Step S15).
For example, in the example illustrated in
As illustrated in
The ultrasound image illustrated in
In a case where it is determined that the confirmation processing is required, by the blood vessel confirmation unit 63, the confirmation processing is executed (Step S16), and the type of the blood vessel, that is, whether the blood vessel is an artery or a vein is confirmed on the basis of the execution result (Step S17).
In a case where it is confirmed whether the blood vessel is an artery or a vein, the graphic including the blood vessel is superimposed on the ultrasound image to be displayed on the monitor 34 on the basis of the confirmation result by the display control unit 33 (Step S18). For example, similar to the example illustrated in
The operation described above is repeatedly performed for each frame of the ultrasound images and for each blood vessel region.
Next, with reference to the flowchart of
First, by the probability calculation unit 61, each of the probability that the pixel is a pixel of an artery, the probability that the pixel is a pixel of a vein, and the probability that the pixel is a pixel of the background other than the blood vessel is calculated for each pixel of the ultrasound image (Step S21). For example, the probability calculation unit 61 calculates each of the probability that the pixel is a pixel of an artery, the probability that the pixel is a pixel of a vein, and the probability that the pixel is a pixel of the background other than the blood vessel by using the machine learning model for the probability estimation.
Here, the machine learning model is a trained model in which learning ultrasound images and ground truth data on whether the pixel is a pixel of an artery, is a pixel of a vein, or is a pixel of the background for each pixel of the learning ultrasound image are used as teacher data, and which has been trained for a relationship between the learning ultrasound images and the ground truth data on whether the pixel is a pixel of an artery, is a pixel of a vein, or is a pixel of the background for each pixel of the learning ultrasound image, for a plurality of pieces of the teacher data.
In this case, in the machine learning model, the ultrasound image of the examination location of the subject at the time of examination is input, and an estimation result of estimating the probability that the pixel is a pixel of an artery, the probability that the pixel is a pixel of a vein, and the probability that the pixel is a pixel of the background is output for each pixel of the ultrasound image.
Then, by the probability calculation unit 61, for each pixel of the ultrasound image, the probability that the pixel is a pixel of an artery, the probability that the pixel is a pixel of a vein, and the probability that the pixel is a pixel of the background are calculated on the basis of the estimation result by the machine learning model.
Subsequently, by the blood vessel determination unit 62, for each pixel of the ultrasound image, the type of the pixel, that is, that the pixel is a pixel of an artery, is a pixel of a vein, or is a pixel of the background is decided on the basis of the probability that the pixel is a pixel of an artery, the probability that the pixel is a pixel of a vein, and the probability that the pixel is a pixel of the background (Step S22).
For example, the blood vessel determination unit 62 decides that the same single pixel is a type of the pixel with the highest probability. That is, in a case where the probability of being a pixel of an artery is highest, it is decided that the pixel is a pixel of an artery. Similarly, in a case where the probability of being a pixel of a vein is highest, it is decided that the pixel is a pixel of a vein, and in a case where the probability of being a pixel of the background is highest, it is decided that the pixel is a pixel of the background.
Subsequently, by the blood vessel determination unit 62, the blood vessel region including the blood vessel is decided on the basis of the pixel of the artery and the pixel of the vein (Step S23). For example, the blood vessel determination unit 62 decides that the region represented by a collection (cluster) of the pixel of the artery and the pixel of the vein is a blood vessel region.
Subsequently, by the blood vessel determination unit 62, whether the blood vessel in the blood vessel region is an artery or a vein, or whether or not the confirmation processing is required is determined on the basis of the area of the pixel of the artery and the area of the pixel of the vein with respect to the area of the blood vessel in the blood vessel region (Step S24).
Here, in a case where a ratio of one of the area of the pixel of the artery or the area of the pixel of the vein to the area of the blood vessel in the blood vessel region is equal to or greater than a third threshold value (Yes in Step S24), it is determined that the blood vessel is an artery or a vein by the blood vessel determination unit 62 (Step S25).
In a case where it is determined that the blood vessel is an artery or a vein, the graphic including the blood vessel is superimposed on the ultrasound image to be displayed on the monitor 34 on the basis of the determination result by the display control unit 33 (Step S26). For example, as illustrated in
In the example illustrated in
For example, it is assumed that the area of the blood vessel in the blood vessel region in the central portion of the ultrasound image illustrated in
In
On the other hand, in a case where a ratio of one of the area of the pixel of the artery or the area of the pixel of the vein to the area of the blood vessel in the blood vessel region is less than the third threshold value (No in Step S24), it is determined that the additional confirmation processing is required by the blood vessel determination unit 62 (Step S27).
For example, in the example illustrated in
For example, it is assumed that the area of the blood vessel in the blood vessel region in the central portion of the ultrasound image illustrated in
The ultrasound image illustrated in
In a case where it is determined that the confirmation processing is required, by the blood vessel confirmation unit 63, the confirmation processing is executed (Step S28), and the type of the blood vessel, that is, whether the blood vessel is an artery or a vein is confirmed on the basis of the execution result (Step S29).
In a case where it is confirmed whether the blood vessel is an artery or a vein, the graphic including the blood vessel is superimposed on the ultrasound image to be displayed on the monitor 34 on the basis of the confirmation result by the display control unit 33 (Step S30). For example, similar to the example illustrated in
Similarly, the operation described above is repeatedly performed for each frame of the ultrasound images and for each blood vessel region.
Next, the confirmation processing will be described. The confirmation processing is not particularly limited, but can be exemplified by a method of pressing the examination location, a method of using a color Doppler mode, and a method of using the ultrasound image of the past frame.
First, the method of pressing the examination location will be described.
In this case, the blood vessel confirmation unit 63 prompts the user to press the examination location of the subject with the ultrasound probe 1. For example, as illustrated in
Subsequently, the blood vessel confirmation unit 63 calculates the aspect ratio of the cross section of the blood vessel on the basis of the ultrasound image generated while the examination location is pressed with the ultrasound probe 1.
Here, in a case where a state where the examination location is not pressed as illustrated in
Then, the blood vessel confirmation unit 63 confirms whether the blood vessel is an artery or a vein on the basis of the aspect ratio of the cross section of the blood vessel. That is, the case where the aspect ratio of the cross section of the blood vessel is not changed means that the blood vessel has not collapsed, the blood vessel confirmation unit 63 confirms that the blood vessel is an artery. Conversely, the case where the aspect ratio of the cross section of the blood vessel is changed means that the blood vessel has collapsed, the blood vessel confirmation unit 63 confirms that the blood vessel is a vein.
By providing a pressure sensor that detects a pressure applied to a contact surface with the examination location of the subject, to the ultrasound probe 1, the blood vessel confirmation unit 63 can determine whether or not the examination location is pressed on the basis of the pressure detected by the pressure sensor.
The blood vessel confirmation unit 63 can detect the motion in the examination location by analyzing the ultrasound images of a plurality of temporally consecutive frames such as the optical flow, and determine whether or not the examination location is pressed on the basis of the motion in the examination location.
Further, the blood vessel confirmation unit 63 can determine the presence or absence of the change in the aspect ratio of the cross section of the blood vessel for a predetermined period of time after prompting the user to press the examination location with the ultrasound probe 1, and determine whether or not the examination location is pressed on the basis of the presence or absence of the change in the aspect ratio of the blood vessel in the predetermined period of time. This determination method does not require the above-described pressure sensor and the image analysis such as the detection of the motion.
In this case, it is preferable that the blood vessel confirmation unit 63 causes the display control unit 33 to display, on the monitor 34, the above-described predetermined period of time, that is, the remaining time for determining the presence or absence of the change in the aspect ratio of the cross section of the blood vessel. Thereby, the user can understand the time during which the examination location is pressed with the ultrasound probe 1, in other words, the timing at which pressing the examination location with the ultrasound probe 1 is ended.
Next, the method of using the color Doppler mode will be described.
In this case, the blood vessel confirmation unit 63 prompts the user to incline the ultrasound probe 1 to the peripheral side or the central side at the examination location of the subject. For example, as illustrated in
Subsequently, the blood vessel confirmation unit 63 confirms whether the blood vessel is an artery or a vein on the basis of the direction of the blood flow in the ultrasound image generated while the ultrasound probe 1 is inclined at the examination location in the color Doppler mode. For example, in a case where the ultrasound probe 1 is inclined to the peripheral side, the blood vessel confirmation unit 63 confirms that the blood flow is an artery in a case where the direction of the blood flow is a direction approaching the ultrasound probe 1, and the blood vessel confirmation unit 63 confirms that the blood vessel is a vein in a case where the direction of the blood flow is a direction away from the ultrasound probe 1. On the other hand, in a case where the ultrasound probe 1 is inclined to the central side, the blood vessel confirmation unit 63 confirms results opposite to the results in a case where the ultrasound probe 1 is inclined to the peripheral side.
Since it is only necessary to determine the direction of the blood flow, it is not essential to display the direction of the blood flow in color in the ultrasound image.
Then, after it is confirmed whether the blood vessel is an artery or a vein, the blood vessel confirmation unit 63 ends the color Doppler mode.
Next, the method of using the ultrasound images of the past frames will be described.
In this case, the blood vessel confirmation unit 63 searches for an ultrasound image of a frame in which the probability that the same blood vessel as the blood vessel in the ultrasound image at the time of examination is an artery or the probability that the same blood vessel is a vein is equal to or greater than a fourth threshold value, among the ultrasound images of the past frames stored in the image memory 32. For example, the blood vessel confirmation unit 63 searches for an ultrasound image of a frame with the highest probability that the same blood vessel is an artery or the highest probability that the same blood vessel is a vein, among the ultrasound images of the past frames.
For example, the blood vessel confirmation unit 63 can detect the blood vessel in the ultrasound images of the adjacent preceding and succeeding frames, and determine that the blood vessels are the same in a case where an overlapping ratio of the blood vessels in the ultrasound images of the preceding and succeeding frames is equal to or greater than the fourth threshold value.
Then, the blood vessel confirmation unit 63 confirms that the blood vessel in the ultrasound image at the time of examination is an artery or a vein on the basis of the probability that the same blood vessel in the ultrasound image of the frame searched from the ultrasound images of the past frames is an artery or the probability that the same blood vessel is a vein. That is, as the probability that the same blood vessel in the ultrasound image of the current frame is an artery or the probability that the same blood vessel is a vein, the probability that the same blood vessel in the ultrasound image of the past frame is an artery or the probability that the same blood vessel is a vein is used as is.
The ultrasound diagnostic apparatus determines the type of the blood vessel on the basis of the probability that the blood vessel is an artery and the probability that the blood vessel is a vein as described above, and in a case where the type of the blood vessel cannot be determined, the confirmation processing is executed to confirm whether the blood vessel is an artery or a vein. Therefore, it is possible to prevent the type of the blood vessel from being erroneously determined, and to greatly improve the determination accuracy of the type of the blood vessel. In addition, since the ultrasound diagnostic apparatus executes the confirmation processing only in a case where the type of the blood vessel cannot be determined, it is possible to greatly suppress the deterioration of the usability.
In a case of puncturing the blood vessel using the ultrasound diagnostic apparatus, the user is required to be skilled in scanning the examination location of the subject and interpreting the ultrasound image, and it is not easy for an unskilled person to scan the examination location and interpret the ultrasound image. Accordingly, by determining the region including the blood vessel and the type of the blood vessel, and displaying the determination result on the monitor 34 as described above, it is possible to assist the user in scanning the examination location and interpreting the ultrasound image.
The display control unit 33 can superimpose the graphic including the blood vessel on the ultrasound image to be displayed on the monitor 34 on the basis of at least one of the determination result on whether the blood vessel is an artery or a vein or the confirmation result on whether the blood vessel is an artery or a vein. That is, the graphic including the blood vessel may be superimposed on the ultrasound image to be displayed on the monitor 34 on the basis of both the determination result and the confirmation result.
In this case, for example, the probability calculation unit 61 calculates a score according to the aspect ratio of the cross section of the blood vessel in the ultrasound image generated while the examination location is pressed with the ultrasound probe 1. Next, the probability calculation unit 61 weights and averages the probability that the blood vessel is an artery and the probability that the blood vessel is a vein, with the score of the aspect ratio of the cross section of the blood vessel to calculate a final score that the blood vessel is an artery and a final score that the blood vessel is a vein.
Then, the display control unit 33 superimposes the final score that the blood vessel is an artery and the final score that the blood vessel is a vein on the ultrasound image to be displayed on the monitor 34, instead of the probability that the blood vessel is an artery and the probability that the blood vessel is a vein. Thereby, the user can know the final score that the blood vessel is an artery and the final score that the blood vessel is a vein which are more accurate than the probability that the blood vessel is an artery and the probability that the blood vessel is a vein.
Similarly, the blood vessel determination unit 62 may determine whether the blood vessel is an artery or a vein or whether or not the confirmation processing is required on the basis of the final score that the blood vessel is an artery and the final score that the blood vessel is a vein instead of the probability that the blood vessel is an artery and the probability that the blood vessel is a vein.
The display control unit 33 may display the same graphic for the same blood vessel on the monitor 34 once whether the blood vessel is an artery or a vein has been decided. Thereby, it is possible to greatly reduce the processing for deciding the type of the blood vessel.
Furthermore, the display control unit 33 may display characters “blood vessel” instead of the graphics, on the monitor 34 in a case where whether the blood vessel is an artery or a vein cannot be decided even by the confirmation processing. Thereby, it is possible for the user to know that there is a blood vessel in the region of the graphic.
Among the kinds of confirmation processing such as the method of pressing the examination location, the method of using the color Doppler mode, and the method of using the ultrasound image of the past frame, two or more kinds of confirmation processing may be used in combination.
Here, the method of pressing the examination location requires the user's operation such as pressing the examination location with the ultrasound probe 1, and the method of using the color Doppler mode requires the user's operation such as inclining the ultrasound probe 1 at the examination location. However, the method of using the ultrasound image of the past frame does not require the user's operation. Therefore, it is desirable to execute the confirmation processing step by step such that, in a case where the method of using the ultrasound image of the past frame is executed first, and the type of the blood vessel cannot be confirmed by the execution result, the method of pressing the examination location, the method of using the color Doppler mode, and the like are sequentially executed.
The present invention is not limited to a stationary ultrasound diagnostic apparatus, and can be similarly applied to a portable ultrasound diagnostic apparatus in which an apparatus main body 3 is realized by a laptop terminal device, and a handheld ultrasound diagnostic apparatus in which an apparatus main body 3 is realized by a handheld terminal device such as a smartphone or a tablet personal computer (PC). The ultrasound probe 1 and the apparatus main body 3 may be connected in a wired or wireless manner. Further, the entire image generation unit 31 or only the signal processing unit 16 may be provided on the ultrasound probe 1 side, or provided on the apparatus main body 3 side.
In the apparatus of the present invention, the hardware configurations of the processing units executing various kinds of processing such as the transmission and reception circuit 14, the image generation unit 31, the display control unit 33, the blood vessel processing unit 35, and the apparatus control unit 36 may be dedicated hardware, or may be various processors or computers that execute programs.
The various processors include a central processing unit (CPU) as a general-purpose processor executing software (program) and functioning as various processing units, a programmable logic device (PLD) as a processor of which the circuit configuration can be changed after manufacturing such as a field programmable gate array (FPGA), and a dedicated electric circuit as a processor having a circuit configuration designed exclusively for executing specific processing such as an application specific integrated circuit (ASIC).
One processing unit may be configured by one of the various processors or may be configured by a combination of the same or different kinds of two or more processors, for example, a combination of a plurality of FPGAs or a combination of an FPGA and a CPU). Further, a plurality of processing units may be configured by one of various processors, or two or more of a plurality of processing units may be collectively configured by using one processor.
For example, there is a form where one processor is configured by a combination of one or more CPUs and software as typified by a computer, such as a server and a client, and this processor functions as a plurality of processing units. Further, there is a form where a processor realizing the functions of the entire system including a plurality of processing units by one integrated circuit (IC) chip as typified by a system on chip (SoC) or the like is used.
Furthermore, the hardware configurations of these various processors are more specifically electric circuitry where circuit elements, such as semiconductor elements, are combined.
The method of the present invention can be carried out, for example, by a program for causing a computer to execute each step of the method. Further, a computer-readable recording medium in which this program is recorded can also be provided.
The present invention has been described in detail, but the present invention is not limited to the above-described embodiments, and various improvements and changes may be made within a range not departing from the scope of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2020-209315 | Dec 2020 | JP | national |
This application is a Continuation of PCT International Application No. PCT/JP2021/039685 filed on Oct. 27, 2021, which claims priority under 35 U.S.C. § 119(a) to Japanese Patent Application No. 2020-209315 filed on Dec. 17, 2020. The above applications are hereby expressly incorporated by reference, in their entirety, into the present application.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2021/039685 | Oct 2021 | US |
| Child | 18321418 | US |
