Patent Grant
11647987
The invention relates to an ultrasound device and method thereof and, more particularly, to an ultrasound device and method thereof capable of generating a panoramic image.
At present, a nidus with a large range can be observed and diagnosed by using ultrasound to capture images within a region of interest repeatedly and the operation depends a lot on professional knowledge and experience of an operator. Since most of ultrasound devices do not have panoramic function, it wastes time and lacks objectivity in usage. Accordingly, an ultrasound panoramic scanning capable of performing comparison for characteristic points in the images and adopting image stitching technology is unique for some applications. Specifically, the accuracy of comparison for characteristic points may be increased by edge comparison of gray-level gradient distribution or by converting information in the images into specific parameters.
The image stitching technology also affects panoramic image quality. The prior art usually performs a blur process on a seam between two images to form a seamless panoramic image. However, the resolution and details of the panoramic image may be damaged. Although the prior art has developed lots of methods for comparing characteristic points, the applicability of the prior art is limited to static tissue, such as muscle tissue, tendon tissue, ligament tissue, and so on. For example, a black-and-white ultrasound scanning is usually used to scan and generate a panoramic image with large area. As long as the scanned part is a static tissue and the subject is kept at a static state, the image will be clear. However, for dynamic tissue, such as carotid artery or blood vessels of limbs, the captured images may have sawtooth shape during the stitching process due to systolic and diastolic blood vessel, such that the panoramic image quality will be unacceptable. However, for carotid atherosclerosis screening, the ultrasound scanning is the simplest and rapidest manner for providing real information for the clinical treatment. Accordingly, how to improve the ultrasound scanning of establishing a panoramic image to improve the panoramic image quality and how to prevent the operator from making a guess and determination according to experience and knowledge are worth researching and improving.
To solve the aforesaid problems, an embodiment of the invention provides a method for generating ultrasonic panoramic image comprising steps of using an ultrasound probe to continuously obtain a plurality of black-and-white images and a plurality of color images from an object; determining a plurality of black-and-white characteristic images from the black-and-white images and determining a plurality of color characteristic images from the color images; stitching the black-and-white characteristic images to form a black-and-white panoramic image and stitching the color characteristic images to forma color panoramic image; and overlaying the color panoramic image with the black-and-white panoramic image to form an output panoramic image.
Another embodiment of the invention provides an ultrasound device comprising an ultrasound probe, a calculating unit and a display unit. The ultrasound probe continuously obtains a plurality of black-and-white images and a plurality of color images from an object. The calculating unit determines a plurality of black-and-white characteristic images from the black-and-white images and determines a plurality of color characteristic images from the color images. The calculating unit stitches the black-and-white characteristic images to forma black-and-white panoramic image and stitches the color characteristic images to form a color panoramic image. The calculating unit overlays the color panoramic image with the black-and-white panoramic image to form an output panoramic image. The display unit displays the output panoramic image.
In the embodiment of the invention, the calculating unit determines whether an overlap index between a first black-and-white image and a second black-and-white image in the black-and-white images satisfies a black-and-white predetermined value; when the overlap index between the first black-and-white image and the second black-and-white image satisfies the black-and-white predetermined value, the calculating unit determines whether an overlap index between a first color image and a second color image in the color images satisfies a color predetermined value; and when the overlap index between the first color image and the second color image satisfies the color predetermined value, the calculating unit sets the first black-and-white image, the second black-and-white image, the first color image and the second color image to be a first black-and-white characteristic image, a second black-and-white characteristic image, a first color characteristic image and a second color characteristic image.
In the embodiment of the invention, the calculating unit generates the overlap index between the first black-and-white image and the second black-and-white image according to a first characteristic algorithm.
In the embodiment of the invention, the calculating unit generates the overlap index between the first color image and the second color image according to a second characteristic algorithm.
In the embodiment of the invention, the calculating unit determines whether an overlap index between the second black-and-white image and a third black-and-white image in the black-and-white images satisfies the black-and-white predetermined value; when the overlap index between the second black-and-white image and the third black-and-white image satisfies the black-and-white predetermined value, the calculating unit determines whether an overlap index between the second color image and a third color image in the color images satisfies the color predetermined value; and when the overlap index between the second color image and the third color image satisfies the color predetermined value, the calculating unit sets the third black-and-white image and the third color image to be a third black-and-white characteristic image and a third color characteristic image.
In the embodiment of the invention, the calculating unit uses a stitching algorithm to stitch the black-and-white characteristic images to form the black-and-white panoramic image and stitch the color characteristic images to form the color panoramic image, and the stitching algorithm comprises one selected from a group consisting of Alpha mixed method, Poisson mixed method and multi-band mixed method.
In the embodiment of the invention, the ultrasound probe continuously obtains a plurality of black-and-white images and a plurality of color images from a first time point and the calculating unit generates a first output panoramic image accordingly; the ultrasound probe continuously obtains a plurality of black-and-white images and a plurality of color images from a second time point and the calculating unit generates a second output panoramic image accordingly.
In the embodiment of the invention, the ultrasound probe continuously obtains a plurality of black-and-white images and a plurality of color images from a plurality of different time points, the calculating unit generates a plurality of output panoramic images accordingly, and the display unit displays the output panoramic images to form a continuous and dynamic panoramic image.
The method and the ultrasound device of the invention at least have the following advantages.
1. No matter what kind and size of the ultrasound probe are, the invention can generate a color Doppler ultrasound panoramic image, save time without necessary to scan and observe repeatedly, and provide overall image information.
2. The invention performs screening and classification according to the information of Doppler signal to reduce error due to systolic and diastolic tissue, such that the ultrasound panoramic image is closer to real human body structure. Accordingly, the invention does not need to use additional sensor to obtain additional reference signal.
3. The invention can observe dynamic and static color Doppler panoramic image simultaneously, such that it is beneficial to observe the blood flow in the blood vessel and observe the panoramic result for a specific moment.
These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.
Certain terms are used throughout the following description and claims to refer to particular components. As one skilled in the art will appreciate, manufacturers may refer to a component by different names. In the following discussion and in the claims, the terms “include” and “comprise” are used in an open-ended fashion. Also, the term “couple” is intended to mean either an indirect or direct electrical/mechanical connection. Thus, if a first device is coupled to a second device, that connection may be through a direct electrical/mechanical connection, or through an indirect electrical/mechanical connection via other devices and connections. Furthermore, the term “black-and-white image” recited in the specification represents “ultrasound B mode image” and the term “color image” represents “ultrasound color mode image” well known by one skilled in the art of ultrasound image process.
Referring to
The ultrasound device 1 of the invention uses the ultrasound probe 10 to move on a surface of the object 100, obtains a series of black-and-white images and a series of corresponding color images in accordance with variations of time and positions, and generates a reliable panoramic image of ultrasound scanning by the two series of image information. To determine characteristic image, the invention compares characteristic points of the black-and-white ultrasound images, compares colors of the corresponding color ultrasound images in different sequences, and determines a series of black-and-white characteristic images and corresponding color characteristic images from the black-and-white images and the corresponding color images. In an embodiment, the ultrasound probe 10 obtains a plurality of color images C from the object 100 by using color Doppler ultrasound mode to perform scanning, which can measure dynamic variation of dynamic tissue including speed or intensity of blood flow and then differentiate the measured speed or intensity of blood flow by colors.
Referring to
Step 202: continuously obtain a plurality of black-and-white images and a plurality of corresponding color images from an object.
Step 204: determine a plurality of black-and-white characteristic images from the black-and-white images and determine a plurality of color characteristic images from the color images.
Step 206: stitch the black-and-white characteristic images to form a black-and-white panoramic image and stitch the color characteristic images to form a color panoramic image.
Step 208: overlay the color panoramic image with the black-and-white panoramic image to form an output panoramic image.
Step 210: display a plurality of output panoramic images generated from a plurality of different time points to form a continuous and dynamic panoramic image.
Referring to
It should be noted that if step 202 is performed from different time points, the output panoramic image obtained in step 208 is different accordingly. Therefore, the invention may utilize the aforesaid characteristic to obtain different output panoramic images from different time points, wherein the output panoramic images generated from a plurality of different time points represent the tissue structures of the human body in different moments. For example, the ultrasound probe 10 continuously obtains a plurality of black-and-white images B and a plurality of corresponding color images C from a first time point (step 202) for generating a first output panoramic image (steps 204-208), continuously obtains a plurality of black-and-white images B and a plurality of corresponding color images C from a second time point (step 202) for generating a second output panoramic image (steps 204-208), and so on. Then, step 210 is performed to output the output panoramic images sequentially, so as to display a continuous and dynamic panoramic image on the display unit 30. Furthermore, the continuous and dynamic panoramic image can be paused to observe the result of the panoramic image at a specific moment.
The process of determining a plurality of black-and-white characteristic images and a plurality of color characteristic images in step 204 is described in the following. Referring to
Step 302: set a first black-and-white image of the black-and-white images to be a black-and-white base image and a black-and-white characteristic image and set a corresponding first color image to be a color base image and a color characteristic image.
Step 304: determine whether the last images of the black-and-white images and the color images have been compared.
Step 306: determine the degree of the overlap area between the black-and-white base image and the follow-up black-and-white image sequentially.
Step 308: determine an image error between the color base image and a color image corresponding to the black-and-white image.
Step 310: set the black-and-white base image and the black-and-white image to be the black-and-white characteristic image and set the color base image and the color image to be the color characteristic image.
Step 312: set the black-and-white image in step 310 to be the black-and-white base image, set the color image in step 310 to be the color base image, and perform step 304.
After obtaining a plurality of black-and-white images B and a plurality of corresponding color images C (as shown in
The black-and-white image B0 (the first black-and-white image), which is served as the black-and-white base image, is then used to be compared with the follow-up each black-and-white image B to determine the degree of the overlap area therebetween, i.e. two input black-and-white images are compared with each other by the iteration manner (sequentially compare the black-and-white image B0 with the black-and-white image B1, compare the black-and-white image B0 with the black-and-white image B2, and so on). For example, in an embodiment, the invention may use enhanced correlation coefficient (ECC) and image registration algorithm to be the first characteristic algorithm to calculate a transition matrix to obtain the overlap area between two black-and-white images. In other embodiments, the invention may use fast feature detector (FAST), scale invariant feature transfer (SIFT), or speeded up robust features (SURF) to be the first characteristic algorithm. After performing the iteration manner again and again, an overlap index (the degree of the overlap area) between the black-and-white image B0 and the follow-up another black-and-white image Bm (second black-and-white image) satisfies a black-and-white predetermined value (step 306), i.e. a sufficient overlap area is between the black-and-white image B0 and the black-and-white image Bm, such that there is sufficient information for comparing the color images in Doppler effect.
Then, by means of an image overlap range obtained according to the black-and-white image B0 and the black-and-white image Bm, a second characteristic algorithm is used to determine an image error between the color image C0 (first color image, which is served as the color base image) with the color image Cm (second color image) corresponding to the black-and-white image Bm (step 308). In practical applications, the invention may determine whether an error (overlap index) between the color image C0 and the color image Cm is low enough, i.e. whether the overlap index also satisfies a color predetermined value. If yes, it means that a Doppler response value of the color image C0 is quite close to a Doppler response value of the color image Cm, i.e. the speed or intensity of the blood flow is quite similar. Accordingly, the black0-and-white image Bm and the corresponding color image Cm can be set to be the black-and-white characteristic image and the color characteristic image (step 310). For example, in an embodiment, the invention may use least square error method to be the second characteristic algorithm to calculate an error between two color images and determine whether the calculated error is lower than the color predetermined value. In other embodiments, the invention may use random sample consensus (RANSAC) and gradient descent to be the second characteristic algorithm.
It should be noted that if the error between the color image C0 and the color image Cm obtained by using the second characteristic algorithm in step 308 is not lower than the color predetermined value, it means that the color image Cm is unsuitable for stitching (the corresponding black-and-white image Bm will also be abandoned). Accordingly, the invention will go back to step 306 to perform the follow-up iteration and comparison. For example, the invention may compare the black-and-white image B0 with the black-and-white image Bm+1 and repeat steps 306, 308 until the aforesaid characteristic images are obtained.
After confirming that the black-and-white image Bm (second black-and-white image) and the color image Cm (second color image) are a black-and-white characteristic image and a color characteristic image, the invention uses the black-and-white image Bm and the color image Cm to be a new black-and-white base image and a new color base image in next cycle and repeats the steps from step 304 and the follow-up image comparison until the last black-and-white image and the last color image are compared. For example, in the second cycle, the black-and-white image Bm (second black-and-white image), which is served as the black-and-white base image, is compared with the follow-up each black-and-white image B sequentially to determine the overlap index until the overlap index between the black-and-white image Bm and the follow-up another black-and-white image Bn (third black-and-white image) satisfies the black-and-white predetermined value (step 306). Then, the invention determines whether the overlap index between the color image Cm (second color image) and a color image Cn (third color image) corresponding to the black-and-white image Bn also satisfies the color predetermined value (step 308). If yes, it means that the black-and-white image Bn and the corresponding color image Cn can be set as the black-and-white characteristic image and the color characteristic image (step 310). Then, the invention uses the black-and-white image Bn (third black-and-white image) and the color image Cn (third color image) to be a new black-and-white base image and a new color base image to perform comparison for the third cycle until the last black-and-white image and the last color image are compared (step 304). Accordingly, the invention can determine a plurality of black-and-white characteristic images and a plurality of corresponding color characteristic images from the captured black-and-white images B and the captured color images C (step 204).
After the aforesaid method, the calculating unit 20 can determine a plurality of black-and-white characteristic images and a plurality of color characteristic images from the black-and-white images B and the corresponding color images C obtained by the ultrasound probe 10. Then, the calculating unit 20 can stitch the black-and-white characteristic images to form a black-and-white panoramic image and stitch the color characteristic images to form a color panoramic image. Then, the calculating unit 20 can overlay the color panoramic image with the black-and-white panoramic image to form a color Doppler panoramic image. Finally, the color output panoramic image can be displayed on the display unit.
The invention proposes a method for generating ultrasonic panoramic image and an ultrasonic device. The invention first analyzes the degree of overlap area between two black-and-white images and determines the Doppler signal error of two corresponding color images once the overlap area is large enough. A plurality of black-and-white characteristic images and color characteristic images are then determined from the captured black-and-white images and color images and used for generating a black-and-white panoramic image and a color panoramic image respectively using an image stitching algorithm. The color panoramic image is then overlaid with the black-and-white panoramic image to form an output panoramic image. Furthermore, a plurality of output panoramic images generated from different time points may further be outputted sequentially to form a continuous and dynamic panoramic image. Accordingly, the ultrasound panoramic scanning may obtain accurate and dynamic image result.
Those skilled in the art will readily observe that numerous modifications and alterations of the device and method may be made while retaining the teachings of the invention. Accordingly, the above disclosure should be construed as limited only by the metes and bounds of the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 107102922 | Jan 2018 | TW | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 6390980 | Peterson | May 2002 | B1 |
| 6540681 | Cheng | Apr 2003 | B1 |
| 20020120195 | Hossack | Aug 2002 | A1 |
| 20050033173 | Von Behren | Feb 2005 | A1 |
| 20210219961 | Toyonaga | Jul 2021 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20190231314 A1 | Aug 2019 | US |
