This invention pertains to the field of obstetrics, particularly to ultrasound-based non-invasive obstetric measurements.
Measurement of the amount of Amniotic Fluid (AF) volume is critical for assessing the kidney and lung function of a fetus and also for assessing the placental function of the mother. Amniotic fluid volume is also a key measure to diagnose conditions such as polyhydramnios (too much AF) and oligohydramnios (too little AF). Polyhydramnios and oligohydramnios are diagnosed in about 7-8% of all pregnancies and these conditions are of concern because they may lead to birth defects or to delivery complications. The amniotic fluid volume is also one of the important components of the fetal biophysical profile, a major indicator of fetal well-being.
The currently practiced and accepted method of quantitatively estimating the AF volume is from two-dimensional (2D) ultrasound images. The most commonly used measure is known as the use of the amniotic fluid index (AFI). AFI is the sum of vertical lengths of the largest AF pockets in each of the 4 quadrants. The four quadrants are defined by the umbilicus (the navel) and the linea nigra (the vertical mid-line of the abdomen). The transducer head is placed on the maternal abdomen along the longitudinal axis with the patient in the supine position. This measure was first proposed by Phelan et al (Phelan J P, Smith C V, Broussard P, Small M., “Amniotic fluid volume assessment with the four-quadrant technique at 36-42 weeks' gestation,” J Reprod Med July; 32(7): 540-2, 1987) and then recorded for a large normal population over time by Moore and Cayle (Moore T R, Cayle J E. “The amniotic fluid index in normal human pregnancy,” Am J Obstet Gynecol May; 162(5): 1168-73, 1990).
Even though the AFI measure is routinely used, studies have shown a very poor correlation of the AFI with the true AF volume (Sepulveda W, Flack N J, Fisk N M., “Direct volume measurement at midtrimester amnioinfusion in relation to ultrasonographic indexes of amniotic fluid volume,” Am J Obstet Gynecol April; 170(4): 1160-3, 1994). The correlation coefficient was found to be as low as 0.55, even for experienced sonographers. The use of vertical diameter only and the use of only one pocket in each quadrant are two reasons why the AFI is not a very good measure of AF Volume (AFV).
Some of the other methods that have been used to estimate AF volume include:
Dye dilution technique. This is an invasive method where a dye is injected into the AF during amniocentesis and the final concentration of dye is measured from a sample of AF removed after several minutes. This technique is the accepted gold standard for AF volume measurement; however, it is an invasive and cumbersome method and is not routinely used.
Subjective interpretation from ultrasound images. This technique is obviously dependent on observer experience and has not been found to be very good or consistent at diagnosing oligo- or poly-hydramnios.
Vertical length of the largest single cord-free pocket. This is an earlier variation of the AFI where the diameter of only one pocket is measured to estimate the AF volume.
Two-diameter areas of the largest AF pockets in the four quadrants. This is similar to the AFI; however, in this case, two diameters are measured instead of only one for the largest pocket. This two diameter area has been recently shown to be better than AFI or the single pocket measurement in identifying oligohydramnios (Magann E F, Perry K G Jr, Chauhan S P, Anfanger P J, Whitworth N S, Morrison J C., “The accuracy of ultrasound evaluation of amniotic fluid volume in singleton pregnancies: the effect of operator experience and ultrasound interpretative technique,” J Clin Ultrasound, June; 25(5):249-53, 1997).
The measurement of various anatomical structures using computational constructs are described, for example, in U.S. Pat. No. 6,346,124 to Geiser, et al. (Autonomous Boundary Detection System For Echocardiographic Images). Similarly, the measurement of bladder structures are covered in U.S. Pat. No. 6,213,949 to Ganguly, et al. (System For Estimating Bladder Volume) and U.S. Pat. No. 5,235,985 to McMorrow, et al., (Automatic Bladder Scanning Apparatus). The measurement of fetal head structures is described in U.S. Pat. No. 5,605,155 to Chalana, et al., (Ultrasound System For Automatically Measuring Fetal Head Size). The measurement of fetal weight is described in U.S. Pat. No. 6,375,616 to Soferman, et al. (Automatic Fetal Weight Determination).
Pertaining to ultrasound-based determination of amniotic fluid volumes, Segiv et al. (in Segiv C, Akselrod S, Tepper R., “Application of a semiautomatic boundary detection algorithm for the assessment of amniotic fluid quantity from ultrasound images.” Ultrasound Med Biol, May; 25(4): 515-26, 1999) describe a method for amniotic fluid segmentation from 2D images. However, the Segiv et al. method is interactive in nature and the identification of amniotic fluid volume is very observer dependent. Moreover, the system described is not a dedicated device for amniotic fluid volume assessment.
Grover et al. (Grover J, Mentakis E A, Ross M G, “Three-dimensional method for determination of amniotic fluid volume in intrauterine pockets.” Obstet Gynecol, December; 90(6): 1007-10, 1997) describe the use of a urinary bladder volume instrument for amniotic fluid volume measurement. The Grover et al. method makes use of the bladder volume instrument without any modifications and uses shape and other anatomical assumptions specific to the bladder that do not generalize to amniotic fluid pockets. Amniotic fluid pockets having shapes not consistent with the Grover et al. bladder model introduces analytical errors. Moreover, the bladder volume instrument does not allow for the possibility of more than one amniotic fluid pocket in one image scan. Therefore, the amniotic fluid volume measurements made by the Grover et al. system may not be correct or accurate.
None of the currently used methods for AF volume estimation are ideal. Therefore, there is a need for better, non-invasive, and easier ways to accurately measure amniotic fluid volume.
The clarity of ultrasound acquired images is affected by motions of the examined subject, the motions of organs and fluids within the examined subject, the motion of the probing ultrasound transceiver, the coupling medium used transceiver and the examined subject, and the algorithms used for image processing. As regards image processing frequency domain approaches have been utilized in the literature including using Wiener filters that is implemented in the frequency domain and assumes that the point spread function (PSF) is fixed and known. This assumption conflicts with the observation that the received ultrasound signals are usually non-stationary and depth-dependent. Since the algorithm is implemented in the frequency domain, the error introduced in PSF will leak across the spatial domain. As a result, the performance of Wiener filtering is not ideal.
As regards prior uses of coupling mediums, the most common container for dispensing ultrasound coupling gel is an 8 oz. plastic squeeze bottle with an open, tapered tip. The tapered tip bottle is inexpensive and easy to refill from a larger reservoir in the form of a bag or pump-type and dispenses gel in a controlled manner. Other embodiments include the Sontac® ultrasound gel pad available from Verathon™ Medical, Bothell, Wash., USA is a pre-packaged, circular pad of moist, flexible coupling gel 2.5 inches in diameter and 0.06 inches thick and is advantageously used with the BladderScan devices. The Sontac pad is simple to apply and to remove, and provides adequate coupling for a one-position ultrasound scan in most cases. Yet others include the Aquaflex® gel pads perform in a similar manner to Sontac pads, but are larger and thicker (2 cm thick×9 cm diameter), and traditionally used for therapeutic ultrasound or where some distance between the probe and the skin surface (“stand-off”) must be maintained.
The main purpose of an ultrasonic coupling medium is to provide an air-free interface between an ultrasound transducer and the body surface. Gels are used as coupling media since they are moist and deformable, but not runny: they wet both the transducer and the body surface, but stay where they are applied. The most common delivery method for ultrasonic coupling gel, the plastic squeeze bottle, has several disadvantages. First, if the bottle has been stored upright the gel will fall to the bottom of the bottle, and vigorous shaking is required to get the gel back to the bottle tip, especially if the gel is cold. This motion can be particularly irritating to sonographers, who routinely suffer from wrist and arm pain from ultrasound scanning. Second, the bottle tip is a two-way valve: squeezing the bottle releases gel at the tip, but releasing the bottle sucks air back into the bottle and into the gel. The presence of air bubbles in the gel may detract from its performance as a coupling medium. Third, there is no standard application amount: inexperienced users such as Diagnostic Ultrasound customers have to make an educated guess about how much gel to use. Fourth, when the squeeze bottle is nearly empty it is next to impossible to coax the final 5-10% of gel into the bottle's tip for dispensing. Finally, although refilling the bottle from a central source is not a particularly difficult task, it is non-sterile and potentially messy.
Sontac pads and other solid gel coupling pads are simpler to use than gel: the user does not have to guess at an appropriate application amount, the pad is sterile, and it can be simply lifted off the patient and disposed of after use. However, pads do not mold to the skin or transducer surface as well as the more liquefied coupling gels and therefore may not provide ideal coupling when used alone, especially on dry, hairy, curved, or wrinkled surfaces. Sontac pads suffer from the additional disadvantage that they are thin and easily damaged by moderate pressure from the ultrasound transducer. (See Bishop S, Draper D O, Knight K L, Feland J B, Eggett D. “Human tissue-temperature rise during ultrasound treatments with the Aquaflex gel pad.” Journal of Athletic Training 39(2):126-131, 2004).
Relating to cannula insertion, unsuccessful insertion and/or removal of a cannula, a needle, or other similar devices into vascular tissue may cause vascular wall damage that may lead to serious complications or even death. Image guided placement of a cannula or needle into the vascular tissue reduces the risk of injury and increases the confidence of healthcare providers in using the foregoing devices. Current image guided placement methods generally use a guidance system for holding specific cannula or needle sizes. The motion and force required to disengage the cannula from the guidance system may, however, contribute to a vessel wall injury, which may result in extravasation. Complications arising from extravasation resulting in morbidity are well documented. Therefore, there is a need for image guided placement of a cannula or needle into vascular tissue while still allowing a health care practitioner to use standard “free” insertion procedures that do not require a guidance system to hold the cannula or needle.
The preferred form of the invention is a three dimensional (3D) ultrasound-based system and method using a hand-held 3D ultrasound device to acquire at least one 3D data set of a uterus and having a plurality of automated processes optimized to robustly locate and measure the volume of amniotic fluid in the uterus without resorting to pre-conceived models of the shapes of amniotic fluid pockets in ultrasound images. The automated process uses a plurality of algorithms in a sequence that includes steps for image enhancement, segmentation, and polishing.
A hand-held 3D ultrasound device is used to image the uterus trans-abdominally. The user moves the device around on the maternal abdomen and, using 2D image processing to locate the amniotic fluid areas, the device gives feedback to the user about where to acquire the 3D image data sets. The user acquires one or more 3D image data sets covering all of the amniotic fluid in the uterus and the data sets are then stored in the device or transferred to a host computer.
The 3D ultrasound device is configured to acquire the 3D image data sets in two formats. The first format is a collection of two-dimensional scanplanes, each scanplane being separated from the other and representing a portion of the uterus being scanned. Each scanplane is formed from one-dimensional ultrasound A-lines confined within the limits of the 2D scanplane. The 3D data sets is then represented as a 3D array of 2D scanplanes. The 3D array of 2D scanplanes is an assembly of scanplanes, and may be assembled into a translational array, a wedge array, or a rotational array.
Alternatively, the 3D ultrasound device is configured to acquire the 3D image data sets from one-dimensional ultrasound A-lines distributed in 3D space of the uterus to form a 3D scancone of 3D-distributed scanline. The 3D scancone is not an assembly of 2D scanplanes.
The 3D image datasets, either as discrete scanplanes or 3D distributed scanlines, are then subjected to image enhancement and analysis processes. The processes are either implemented on the device itself or is implemented on the host computer. Alternatively, the processes can also be implemented on a server or other computer to which the 3D ultrasound data sets are transferred.
In a preferred image enhancement process, each 2D image in the 3D dataset is first enhanced using non-linear filters by an image pre-filtering step. The image pre-filtering step includes an image-smoothing step to reduce image noise followed by an image-sharpening step to obtain maximum contrast between organ wall boundaries.
A second process includes subjecting the resulting image of the first process to a location method to identify initial edge points between amniotic fluid and other fetal or maternal structures. The location method automatically determines the leading and trailing regions of wall locations along an A-mode one-dimensional scan line.
A third process includes subjecting the image of the first process to an intensity-based segmentation process where dark pixels (representing fluid) are automatically separated from bright pixels (representing tissue and other structures).
In a fourth process, the images resulting from the second and third step are combined to result in a single image representing likely amniotic fluid regions.
In a fifth process, the combined image is cleaned to make the output image smooth and to remove extraneous structures such as the fetal head and the fetal bladder.
In a sixth process, boundary line contours are placed on each 2D image. Thereafter, the method then calculates the total 3D volume of amniotic fluid in the uterus.
In cases in which uteruses are too large to fit in a single 3D array of 2D scanplanes or a single 3D scancone of 3D distributed scanlines, especially as occurs during the second and third trimester of pregnancy, preferred alternate embodiments of the invention allow for acquiring at least two 3D data sets, preferably four, each 3D data set having at least a partial ultrasonic view of the uterus, each partial view obtained from a different anatomical site of the patient.
In one embodiment a 3D array of 2D scanplanes is assembled such that the 3D array presents a composite image of the uterus that displays the amniotic fluid regions to provide the basis for calculation of amniotic fluid volumes. In a preferred alternate embodiment, the user acquires the 3D data sets in quarter sections of the uterus when the patient is in a supine position. In this 4-quadrant supine procedure, four image cones of data are acquired near the midpoint of each uterine quadrant at substantially equally spaced intervals between quadrant centers. Image processing as outlined above is conducted for each quadrant image, segmenting on the darker pixels or voxels associated with amniotic fluid. Correcting algorithms are applied to compensate for any quadrant-to-quadrant image cone overlap by registering and fixing one quadrant's image to another. The result is a fixed 3D mosaic image of the uterus and the amniotic fluid volumes or regions in the uterus from the four separate image cones.
Similarly, in another preferred alternate embodiment, the user acquires one or more 3D image data sets of quarter sections of the uterus when the patient is in a lateral position. In this multi-image cone lateral procedure, each image cones of data are acquired along a lateral line of substantially equally spaced intervals. Each image cone are subjected to the image processing as outlined above, with emphasis given to segmenting on the darker pixels or voxels associated with amniotic fluid. Scanplanes showing common pixel or voxel overlaps are registered into a common coordinate system along the lateral line. Correcting algorithms are applied to compensate for any image cone overlap along the lateral line. The result is a fixed 3D mosaic image of the uterus and the amniotic fluid volumes or regions in the uterus from the four separate image cones.
In yet other preferred embodiments, at least two 3D scancone of 3D distributed scanlines are acquired at different anatomical sites, image processed, registered and fused into a 3D mosaic image composite. Amniotic fluid volumes are then calculated.
The system and method further provides an automatic method to detect and correct for any contribution the fetal head provides to the amniotic fluid volume.
Systems, methods, and devices for image clarity of ultrasound-based images are described. Such systems, methods, and devices include improved transducer aiming and utilizing time-domain deconvolution processes upon the non-stationary effects of ultrasound signals. The processes deconvolution applies algorithms to improve the clarity or resolution of ultrasonic images by suppressed reverberation of ultrasound echoes. The initially acquired and distorted ultrasound image is reconstructed to a clearer image by countering the effect of distortion operators. An improved point spread function (PSF) of the imaging system is applied, utilizing a deconvolution algorithm, to improve the image resolution, and remove reverberations by modeling them as noise.
As regards improved transducer aiming particular embodiments employ novel applications of computer vision techniques to perform real time analysis. First, a computer vision method is introduced: optical flow, which is a powerful motion analysis technique and is applied in many different research or commercial fields. The optical flow is able to estimate the velocity field of image series and the velocity vector provides information of the contents inside the image series. In the current field, if the target is with very large motion and the motion is in a specific pattern, like moving orientation, the velocity information inside and around the target can be different from other parts in the field. Otherwise, there will be no valuable information in current field and the scanning has to be adjusted.
As regards analyzing the motions of organ movement and fluid flows within an examined subject, new optical-flow-based methods for estimating heart motion from two-dimensional echocardiographic sequences, an optical-flow guided active contour method for Myocardial tracking in contrast echocardiography, and a method for shape-driven segmentation and tracking of the left ventricle.
As regards cannula insertion, ultrasound motion of the cannula is configured by cannula fitted with echogenic ultrasound micro reflectors.
As regards sonic coupling gel media to improve ultrasound communication between a transducer and the examined subject, embodiments include an apparatus that: dispenses a metered quantity of ultrasound coupling gel and enables one-handed gel application. The apparatus also preserves the gel in a de-gassed state (no air bubbles), preserves the gel in a sterile state (no contact between gel applicator and patient), includes a method for easy container refill, and preserves the shape and volume of existing gel application bottles.
The preferred portable embodiment of the ultrasound transceiver of the amniotic fluid volume measuring system are shown in
The top button 16 selects for different acquisition volumes. The transceiver is controlled by a microprocessor and software associated with the microprocessor and a digital signal processor of a computer system. As used in this invention, the term “computer system” broadly comprises any microprocessor-based or other computer system capable of executing operating instructions and manipulating data, and is not limited to a traditional desktop or notebook computer. The display 24 presents alphanumeric or graphic data indicating the proper or optimal positioning of the transceiver 10 for initiating a series of scans. The transceiver 10 is configured to initiate the series of scans to obtain and present 3D images as either a 3D array of 2D scanplanes or as a single 3D scancone of 3D distributed scanlines. A suitable transceiver is the DCD372 made by Diagnostic Ultrasound. In alternate embodiments, the two- or three-dimensional image of a scan plane may be presented in the display 24.
Although the preferred ultrasound transceiver is described above, other transceivers may also be used. For example, the transceiver need not be battery-operated or otherwise portable, need not have a top-mounted display 24, and may include many other features or differences. The display 24 may be a liquid crystal display (LCD), a light emitting diode (LED), a cathode ray tube (CRT), or any suitable display capable of presenting alphanumeric data or graphic images.
Each amniotic fluid volume measuring systems includes the transceiver 10 for acquiring data from a patient. The transceiver 10 is placed in the cradle 52 to establish signal communication with the computer 52. Signal communication as illustrated is by a wired connection from the cradle 42 to the computer 52. Signal communication between the transceiver 10 and the computer 52 may also be by wireless means, for example, infrared signals or radio frequency signals. The wireless means of signal communication may occur between the cradle 42 and the computer 52, the transceiver 10 and the computer 52, or the transceiver 10 and the cradle 42.
A preferred first embodiment of the amniotic fluid volume measuring system includes each transceiver 10 being separately used on a patient and sending signals proportionate to the received and acquired ultrasound echoes to the computer 52 for storage. Residing in each computer 52 are imaging programs having instructions to prepare and analyze a plurality of one dimensional (1D) images from the stored signals and transforms the plurality of 1D images into the plurality of 2D scanplanes. The imaging programs also present 3D renderings from the plurality of 2D scanplanes. Also residing in each computer 52 are instructions to perform the additional ultrasound image enhancement procedures, including instructions to implement the image processing algorithms.
A preferred second embodiment of the amniotic fluid volume measuring system is similar to the first embodiment, but the imaging programs and the instructions to perform the additional ultrasound enhancement procedures are located on the server 56. Each computer 52 from each amniotic fluid volume measuring system receives the acquired signals from the transceiver 10 via the cradle 51 and stores the signals in the memory of the computer 52. The computer 52 subsequently retrieves the imaging programs and the instructions to perform the additional ultrasound enhancement procedures from the server 56. Thereafter, each computer 52 prepares the 1D images, 2D images, 3D renderings, and enhanced images from the retrieved imaging and ultrasound enhancement procedures. Results from the data analysis procedures are sent to the server 56 for storage.
A preferred third embodiment of the amniotic fluid volume measuring system is similar to the first and second embodiments, but the imaging programs and the instructions to perform the additional ultrasound enhancement procedures are located on the server 56 and executed on the server 56. Each computer 52 from each amniotic fluid volume measuring system receives the acquired signals from the transceiver 10 and via the cradle 51 sends the acquired signals in the memory of the computer 52. The computer 52 subsequently sends the stored signals to the server 56. In the server 56, the imaging programs and the instructions to perform the additional ultrasound enhancement procedures are executed to prepare the 1D images, 2D images, 3D renderings, and enhanced images from the server 56 stored signals. Results from the data analysis procedures are kept on the server 56, or alternatively, sent to the computer 52.
As the scanlines are transmitted and received, the returning echoes are interpreted as analog electrical signals by a transducer, converted to digital signals by an analog-to-digital converter, and conveyed to the digital signal processor of the computer system for storage and analysis to determine the locations of the amniotic fluid walls. The computer system is representationally depicted in
The internal scanlines are represented by scanlines 312A-C. The number and location of the internal scanlines emanating from the transceiver 10 is the number of internal scanlines needed to be distributed within the scancone 300, at different positional coordinates, to sufficiently visualize structures or images within the scancone 300. The internal scanlines are not peripheral scanlines. The peripheral scanlines are represented by scanlines 314A-F and occupy the conic periphery, thus representing the peripheral limits of the scancone 300.
Based on fetal position information acquired from data gathered under continuous acquisition mode, the patient is placed in a lateral recumbent position such that the fetus is displaced towards the ground creating a large pocket of amniotic fluid close to abdominal surface where the transceiver 10 can be placed as shown in
After the patient has been placed in the desired position, the transceiver 10 is again operated in the 2D continuous acquisition mode and is moved around on the lateral surface of the patient's abdomen. The operator finds the location that shows the largest amniotic fluid area based on acquiring the largest dark region imaged and the largest alphanumeric value displayed on the display 24. At the lateral abdominal location providing the largest dark region, the transceiver 10 is held in a fixed position, the trigger 14 is released to acquire a 3D image comprising a set of arrayed scanplanes. The 3D image presents a rotational array of the scanplanes 210 similar to the 3D array 240.
In a preferred alternate data acquisition protocol, the operator can reposition the transceiver 10 to different abdominal locations to acquire new 3D images comprised of different scanplane arrays similar to the 3D array 240. Multiple scan cones obtained from different positions provide the operator the ability to image the entire amniotic fluid region from different view points. In the case of a single image cone being too small to accommodate a large AFV measurement, obtaining multiple 3D array 240 image cones ensures that the total volume of large AFV regions is determined. Multiple 3D images may also be acquired by pressing the top bottom 16 to select multiple conic arrays similar to the 3D array 240.
Depending on the position of the fetus relative to the location of the transceiver 10, a single image scan may present an underestimated volume of AFV due to amniotic fluid pockets that remain hidden behind the limbs of the fetus. The hidden amniotic fluid pockets present as unquantifiable shadow-regions.
To guard against underestimating AFV, repeated positioning the transceiver 10 and rescanning can be done to obtain more than one ultrasound view to maximize detection of amniotic fluid pockets. Repositioning and rescanning provides multiple views as a plurality of the 3D arrays 240 images cones. Acquiring multiple images cones improves the probability of obtaining initial estimates of AFV that otherwise could remain undetected and un-quantified in a single scan.
In an alternative scan protocol, the user determines and scans at only one location on the entire abdomen that shows the maximum amniotic fluid area while the patient is the supine position. As before, when the user presses the top button 16, 2D scanplane images equivalent to the scanplane 210 are continuously acquired and the amniotic fluid area on every image is automatically computed. The user selects one location that shows the maximum amniotic fluid area. At this location, as the user releases the scan button, a full 3D data cone is acquired and stored in the device's memory.
The algorithms expressed in 2D terms are used during the targeting phase where the operator trans-abdominally positions and repositions the transceiver 10 to obtain real-time feedback about the amniotic fluid area in each scanplane. The algorithms expressed in 3D terms are used to obtain the total amniotic fluid volume computed from the voxels contained within the calculated amniotic fluid regions in the 3D conic array 240.
The enhancement, segmentation and polishing algorithms depicted in
Other preferred embodiments of the enhancement, segmentation and polishing algorithms depicted in
The enhancement, segmentation and polishing algorithms depicted in
where u is the image being processed. The image u is 2D, and is comprised of an array of pixels arranged in rows along the x-axis, and an array of pixels arranged in columns along the y-axis. The pixel intensity of each pixel in the image u has an initial input image pixel intensity (I) defined as u0=I. The value of I depends on the application, and commonly occurs within ranges consistent with the application. For example, I can be as low as 0 to 1, or occupy middle ranges between 0 to 127 or 0 to 512. Similarly, I may have values occupying higher ranges of 0 to 1024 and 0 to 4096, or greater. The heat equation E1 results in a smoothing of the image and is equivalent to the Gaussian filtering of the image. The larger the number of iterations that it is applied for the more the input image is smoothed or blurred and the more the noise that is reduced.
The shock filter 518 is a PDE used to sharpen images as detailed below. The two dimensional shock filter E2 is expressed as:
where u is the image processed whose initial value is the input image pixel intensity (I): u0=I where the l(u) term is the Laplacian of the image u, F is a function of the Laplacian, and ∥∇u∥ is the 2D gradient magnitude of image intensity defined by equation E3.
∥∇u∥=√{square root over (ux2+uy2)}, E3
l(u)=uxxux2+2uxyuxuy+uyyuy2 E4
of u along the x-axis,
of u along the y-axis,
of u along the x-axis,
of u along the y-axis,
of u along the x-axis,
of u along the y-axis,
of u along the x and y axes, and
The combination of heat filtering and shock filtering produces an enhanced image ready to undergo the intensity-based and edge-based segmentation algorithms as discussed below.
1. Initially determine or categorize cluster boundaries by defining a minimum and a maximum pixel intensity value for every white, gray, or black pixels into groups or k-clusters that are equally spaced in the entire intensity range.
2. Assign each pixel to one of the white, gray or black k-clusters based on the currently set cluster boundaries.
3. Calculate a mean intensity for each pixel intensity k-cluster or group based on the current assignment of pixels into the different k-clusters. The calculated mean intensity is defined as a cluster center. Thereafter, new cluster boundaries are determined as mid points between cluster centers.
4. Determine if the cluster boundaries significantly change locations from their previous values. Should the cluster boundaries change significantly from their previous values, iterate back to step 2, until the cluster centers do not change significantly between iterations. Visually, the clustering process is manifest by the segmented image and repeated iterations continue until the segmented image does not change between the iterations.
The pixels in the cluster having the lowest intensity value—the darkest cluster—are defined as pixels associated with amniotic fluid. For the 2D algorithm, each image is clustered independently of the neighboring images. For the 3D algorithm, the entire volume is clustered together. To make this step faster, pixels are sampled at 2 or any multiple sampling rate factors before determining the cluster boundaries. The cluster boundaries determined from the down-sampled data are then applied to the entire data.
The spatial gradient 526 computes the x-directional and y-directional spatial gradients of the enhanced image. The Hysteresis threshold 530 algorithm detects salient edges. Once the edges are detected, the regions defined by the edges are selected by a user employing the ROI 534 algorithm to select regions-of-interest deemed relevant for analysis.
Since the enhanced image has very sharp transitions, the edge points can be easily determined by taking x- and y-derivatives using backward differences along x- and y-directions. The pixel gradient magnitude ∥∇I∥ is then computed from the x- and y-derivative image in equation E5 as:
∥∇I∥=√{square root over (Ix2+Iy2)} E5
Where I2x=the square of x-derivative of intensity; and
Significant edge points are then determined by thresholding the gradient magnitudes using a hysteresis thresholding operation. Other thresholding methods could also be used. In hysteresis thresholding 530, two threshold values, a lower threshold and a higher threshold, are used. First, the image is thresholded at the lower threshold value and a connected component labeling is carried out on the resulting image. Next, each connected edge component is preserved which has at least one edge pixel having a gradient magnitude greater than the upper threshold. This kind of thresholding scheme is good at retaining long connected edges that have one or more high gradient points.
In the preferred embodiment, the two thresholds are automatically estimated. The upper gradient threshold is estimated at a value such that at most 97% of the image pixels are marked as non-edges. The lower threshold is set at 50% of the value of the upper threshold. These percentages could be different in different implementations. Next, edge points that lie within a desired region-of-interest are selected 534. This region of interest selection 534 excludes points lying at the image boundaries and points lying too close to or too far from the transceiver 10. Finally, the matching edge filter 538 is applied to remove outlier edge points and fill in the area between the matching edge points.
The edge-matching algorithm 538 is applied to establish valid boundary edges and remove spurious edges while filling the regions between boundary edges. Edge points on an image have a directional component indicating the direction of the gradient. Pixels in scanlines crossing a boundary edge location will exhibit two gradient transitions depending on the pixel intensity directionality. Each gradient transition is given a positive or negative value depending on the pixel intensity directionality. For example, if the scanline approaches an echo reflective bright wall from a darker region, then an ascending transition is established as the pixel intensity gradient increases to a maximum value, i.e., as the transition ascends from a dark region to a bright region. The ascending transition is given a positive numerical value. Similarly, as the scanline recedes from the echo reflective wall, a descending transition is established as the pixel intensity gradient decreases to or approaches a minimum value. The descending transition is given a negative numerical value.
Valid boundary edges are those that exhibit ascending and descending pixel intensity gradients, or equivalently, exhibit paired or matched positive and negative numerical values. The valid boundary edges are retained in the image. Spurious or invalid boundary edges do not exhibit paired ascending-descending pixel intensity gradients, i.e., do not exhibit paired or matched positive and negative numerical values. The spurious boundary edges are removed from the image.
For amniotic fluid volume related applications, most edge points for amniotic fluid surround a dark, closed region, with directions pointing inwards towards the center of the region. Thus, for a convex-shaped region, the direction of a gradient for any edge point, the edge point having a gradient direction approximately opposite to the current point represents the matching edge point. Those edge points exhibiting an assigned positive and negative value are kept as valid edge points on the image because the negative value is paired with its positive value counterpart. Similarly, those edge point candidates having unmatched values, i.e., those edge point candidates not having a negative-positive value pair, are deemed not to be true or valid edge points and are discarded from the image.
The matching edge point algorithm 538 delineates edge points not lying on the boundary for removal from the desired dark regions. Thereafter, the region between any two matching edge points is filled in with non-zero pixels to establish edge-based segmentation. In a preferred embodiment of the invention, only edge points whose directions are primarily oriented co-linearly with the scanline are sought to permit the detection of matching front wall and back wall pairs.
Returning to
Upon completion of the AND Operator of Images 442 algorithm, the polish 464 algorithm of
Closing and opening algorithms are operations that process images based on the knowledge of the shape of objects contained on a black and white image, where white represents foreground regions and black represents background regions. Closing serves to remove background features on the image that are smaller than a specified size. Opening serves to remove foreground features on the image that are smaller than a specified size. The size of the features to be removed is specified as an input to these operations. The opening algorithm 550 removes unlikely amniotic fluid regions from the segmented image based on a-priori knowledge of the size and location of amniotic fluid pockets.
Referring to
The AdAFA and AdAVA values obtained by the Close 546 algorithm are reduced by the morphological opening algorithm 550. Thereafter, the AdAFA and AdAVA values are further reduced by removing areas and volumes attributable to deep regions by using the Remove Deep Regions 554 algorithm. Thereafter, the polishing algorithm 464 continues by applying a fetal head region detection algorithm 560.
Fetal brain tissue has substantially similar ultrasound echo qualities as presented by amniotic fluid. If not detected and subtracted from amniotic fluid volumes, fetal brain tissue volumes will be measured as part of the total amniotic fluid volumes and lead to an overestimation and false diagnosis of oligo or poly-hyraminotic conditions. Thus detecting fetal head position, measuring fetal brain matter volumes, and deducting the fetal brain matter volumes from the amniotic fluid volumes to obtain a corrected amniotic fluid volume serves to establish accurately measure amniotic fluid volumes.
The gestational age input 726 begins the fetal head detection algorithm 560 and uses a head dimension table to obtain ranges of head bi-parietal diameters (BPD) to search for (e.g., 30 week gestational age corresponds to a 6 cm head diameter). The head diameter range is input to both the Head Edge Detection, 734, and the Hough Transform, 736. The head edge detection 734 algorithm seeks out the distinctively bright ultrasound echoes from the anterior and posterior walls of the fetal skull while the Hough Transform algorithm, 736, finds the fetal head using circular shapes as models for the fetal head in the Cartesian image (pre-scan conversion to polar form).
Scanplanes processed by steps 522, 538, 530, are input to the head edge detection step 734. Applied as the first step in the fetal head detection algorithm 734 is the detection of the potential head edges from among the edges found by the matching edge filter. The matching edge 538 filter outputs pairs of edge points potentially belonging to front walls or back walls. Not all of these walls correspond to fetal head locations. The edge points representing the fetal head are determined using the following heuristics:
The pixels found satisfying these features are then vertically dilated to produce a set of thick fetal head edges as the output of Head Edge Detection, 734.
The coordinates of a circle in the Cartesian space (x,y) with center (x0,y0) and radius R are defined for an angle θ are derived and defined in equation E5 as:
x=R cos θ+x0
y=R sin θ+y0
(x−x0)2+(y−y0)2=R2 E5
In polar space, the coordinates (r,φ), with respect to the center (r0,φ0), are derived and defined in equation E6 as:
r sin φ=R cos θ+r0 sin φ0
r cos φ=R sin θ+r0 cos φ0
(r sin φ−r0 sin φ0)2+(r cos φ−r0 cos φ0)2=R2 E6
The Hough transform 736 algorithm using equations E5 and E6 attempts to find the best-fit circle to the edges of an image. A circle in the polar space is defined by a set of three parameters, (r0,φ0, R) representing the center and the radius of the circle.
The basic idea for the Hough transform 736 is as follows. Suppose a circle is sought having a fixed radius (say, R1) for which the best center of the circle is similarly sought. Now, every edge point on the input image lies on a potential circle whose center lays R1 pixels away from it. The set of potential centers themselves form a circle of radius R1 around each edge pixel. Now, drawing potential circles of radius R1 around each edge pixel, the point at which most circles intersect, a center of the circle that represents a best-fit circle to the given edge points is obtained. Therefore, each pixel in the Hough transform output contains a likelihood value that is simply the count of the number of circles passing through that point.
This search for best fitting circles can be easily extended to circles with varying radii by adding one more degree of freedom—however, a discrete set of radii around the mean radii for a given gestational age makes the search significantly faster, as it is not necessary to search all possible radii.
The next step in the head detection algorithm is selecting or rejecting best-fit circles based on its likelihood, in the find maximum Hough Value 742 algorithm. The greater the number of circles passing through a given point in the Hough-space, the more likely it is to be the center of a best-fit circle. A 2D metric as a maximum Hough value 742 of the Hough transform 736 output is defined for every image in a dataset. The 3D metric is defined as the maximum of the 2D metrics for the entire 3D dataset. A fetal head is selected on an image depending on whether its 3D metric value exceeds a preset 3D threshold and also whether the 2D metric exceeds a preset 2D threshold. The 3D threshold is currently set at 7 and the 2D threshold is currently set at 5. These thresholds have been determined by extensive training on images where the fetal head was known to be present or absent.
Thereafter, the fetal head detection algorithm concludes with a fill circle region 746 that incorporates pixels to the image within the detected circle. The fill circle region 746 algorithm fills the inside of the best fitting polar circle. Accordingly, the fill circle region 746 algorithm encloses and defines the area of the fetal brain tissue, permitting the area and volume to be calculated and deducted via algorithm 554 from the apparent amniotic fluid area and volume (AAFA or AAFV) to obtain a computation of the corrected amniotic fluid area or volume via algorithm 484.
An example output of applying the head edge detection 734 algorithm to detect potential head edges is shown in image 930. Occupying the space between the anterior and posterior walls are dilated black pixels 932 (stacks or short lines of black pixels representing thick edges). An example of the polar Hough transform 738 for one actual data sample for a specific radius is shown in polar coordinate image 940.
An example of the best-fit circle on real data polar data is shown in polar coordinate image 950 that has undergone the find maximum Hough value step 742. The polar coordinate image 950 is scan-converted to a Cartesian data in image 960 where the effects of finding maximum Hough value 742 algorithm are seen in Cartesian format.
After the contours on all the images have been delineated, the volume of the segmented structure is computed. Two specific techniques for doing so are disclosed in detail in U.S. Pat. No. 5,235,985 to McMorrow et al, herein incorporated by reference. This patent provides detailed explanations for non-invasively transmitting, receiving and processing ultrasound for calculating volumes of anatomical structures.
Multiple Image Cone Acquisition and Image Processing Procedures:
In some embodiments, multiple cones of data acquired at multiple anatomical sampling sites may be advantageous. For example, in some instances, the pregnant uterus may be too large to completely fit in one cone of data sampled from a single measurement or anatomical site of the patient (patient location). That is, the transceiver 10 is moved to different anatomical locations of the patient to obtain different 3D views of the uterus from each measurement or transceiver location.
Obtaining multiple 3D views may be especially needed during the third trimester of pregnancy, or when twins or triplets are involved. In such cases, multiple data cones can be sampled from different anatomical sites at known intervals and then combined into a composite image mosaic to present a large uterus in one, continuous image. In order to make a composite image mosaic that is anatomically accurate without duplicating the anatomical regions mutually viewed by adjacent data cones, ordinarily it is advantageous to obtain images from adjacent data cones and then register and subsequently fuse them together. In a preferred embodiment, to acquire and process multiple 3D data sets or images cones, at least two 3D image cones are generally preferred, with one image cone defined as fixed, and the other image cone defined as moving.
The 3D image cones obtained from each anatomical site may be in the form of 3D arrays of 2D scanplanes, similar to the 3D array 240. Furthermore, the 3D image cone may be in the form of a wedge or a translational array of 2D scanplanes. Alternatively, the 3D image cone obtained from each anatomical site may be a 3D scancone of 3D-distributed scanlines, similar to the scancone 300.
The term “registration” with reference to digital images means the determination of a geometrical transformation or mapping that aligns viewpoint pixels or voxels from one data cone sample of the object (in this embodiment, the uterus) with viewpoint pixels or voxels from another data cone sampled at a different location from the object. That is, registration involves mathematically determining and converting the coordinates of common regions of an object from one viewpoint to the coordinates of another viewpoint. After registration of at least two data cones to a common coordinate system, the registered data cone images are then fused together by combining the two registered data images by producing a reoriented version from the view of one of the registered data cones. That is, for example, a second data cone's view is merged into a first data cone's view by translating and rotating the pixels of the second data cone's pixels that are common with the pixels of the first data cone. Knowing how much to translate and rotate the second data cone's common pixels or voxels allows the pixels or voxels in common between both data cones to be superimposed into approximately the same x, y, z, spatial coordinates so as to accurately portray the object being imaged. The more precise and accurate the pixel or voxel rotation and translation, the more precise and accurate is the common pixel or voxel superimposition or overlap between adjacent image cones. The precise and accurate overlap between the images assures the construction of an anatomically correct composite image mosaic substantially devoid of duplicated anatomical regions.
To obtain the precise and accurate overlap of common pixels or voxels between the adjacent data cones, it is advantageous to utilize a geometrical transformation that substantially preserves most or all distances regarding line straightness, surface planarity, and angles between the lines as defined by the image pixels or voxels. That is, the preferred geometrical transformation that fosters obtaining an anatomically accurate mosaic image is a rigid transformation that doesn't permit the distortion or deforming of the geometrical parameters or coordinates between the pixels or voxels common to both image cones.
The preferred rigid transformation first converts the polar coordinate scanplanes from adjacent image cones into in x, y, z Cartesian axes. After converting the scanplanes into the Cartesian system, a rigid transformation, T, is determined from the scanplanes of adjacent image cones having pixels in common. The transformation T is a combination of a three-dimensional translation vector expressed in Cartesian as t=(Tx, Ty, Tz), and a three-dimensional rotation R matrix expressed as a function of Euler angles θx, θy, θz around the x, y, and z axes. The transformation represents a shift and rotation conversion factor that aligns and overlaps common pixels from the scanplanes of the adjacent image cones.
In the preferred embodiment of the present invention, the common pixels used for the purposes of establishing registration of three-dimensional images are the boundaries of the amniotic fluid regions as determined by the amniotic fluid segmentation algorithm described above.
Several different protocols may be used to collect and process multiple cones of data from more than one measurement site are described in
The preferred embodiment for making a composite image mosaic involves obtaining four multiple image cones where the transceiver 10 is placed at four measurement sites over the patient in a supine or lateral position such that at least a portion of the uterus is ultrasonically viewable at each measurement site. The first measurement site is originally defined as fixed, and the second site is defined as moving and placed at a first known inter-site distance relative to the first site. The second site images are registered and fused to the first site images After fusing the second site images to the first site images, the third measurement site is defined as moving and placed at a second known inter-site distance relative to the fused second site now defined as fixed. The third site images are registered and fused to the second site images Similarly, after fusing the third site images to the second site images, the fourth measurement site is defined as moving and placed at a third known inter-site distance relative to the fused third site now defined as fixed. The fourth site images are registered and fused to the third site images
The four measurement sites may be along a line or in an array. The array may include rectangles, squares, diamond patterns, or other shapes. Preferably, the patient is positioned such that the baby moves downward with gravity in the uterus and displaces the amniotic fluid upwards toward the measuring positions of the transceiver 10.
The interval or distance between each measurement site is approximately equal, or may be unequal. For example in the lateral protocol, the second site is spaced approximately 6 cm from the first site, the third site is spaced approximately 6 cm from the second site, and the fourth site is spaced approximately 6 cm from the third site. The spacing for unequal intervals could be, for example, the second site is spaced approximately 4 cm from the first site, the third site is spaced approximately 8 cm from the second site, and the third is spaced approximately 6 cm from the third site. The interval distance between measurement sites may be varied as long as there are mutually viewable regions of portions of the uterus between adjacent measurement sites.
For uteruses not as large as requiring four measurement sites, two and three measurement sites may be sufficient for making a composite 3D image mosaic. For three measurement sites, a triangular array is possible, with equal or unequal intervals. Furthermore, is the case when the second and third measurement sites have mutually viewable regions from the first measurement site, the second interval may be measured from the first measurement site instead of measuring from the second measurement site.
For very large uteruses not fully captured by four measurement or anatomical sites, greater than four measurement sites may be used to make a composite 3D image mosaic provided that each measurement site is ultrasonically viewable for at least a portion of the uterus. For five measurement sites, a pentagon array is possible, with equal or unequal intervals. Similarly, for six measurement sites, a hexagon array is possible, with equal or unequal intervals between each measurement site. Other polygonal arrays are possible with increasing numbers of measurement sites.
The geometrical relationship between each image cone must be ascertained so that overlapping regions can be identified between any two image cones to permit the combining of adjacent neighboring cones so that a single 3D mosaic composite image is produced from the 4-quadrant or in-line laterally acquired images.
The translational and rotational adjustments of each moving cone to conform with the voxels common to the stationary image cone is guided by an inputted initial transform that has the expected translational and rotational values. The distance separating the transceiver 10 between image cone acquisitions predicts the expected translational and rotational values. For example, as shown in
Next, the known initial transform 1136, for example, (6, 0, 0) for the Cartesian Tx, Ty, Tz terms and (0, 0, 0) for the θx, θy, θz Euler angle terms for an inter-transceiver interval of 6 cm, is subsequently applied to the moving image by the Apply Transform 1140 step. This transformed image is then compared to the fixed image to examine for the quantitative occurrence of overlapping voxels. If the overlap is less than 20%, there are not enough common voxels available for registration and the initial transform is considered sufficient for fusing at step 1016.
If the overlapping voxel sets by the initial transform exceed 20% of the fixed image p voxel sets, the q-voxels of the initial transform are subjected to an iterative sequence of rigid registration.
A transformation T serves to register a first voxel point set p from the first image cone by merging or overlapping a second voxel point set q from a second image cone that is common to p of the first image cone. A point in the first voxel point set p may be defined as pi=(xi, yi, zi) and a point in the second voxel point set q may similarly be defined as qj=(xj, yj, zj), If the first image cone is considered to be a fixed landmark, then the T factor is applied to align (translate and rotate) the moving voxel point set q onto the fixed voxel point set p.
The precision of T is often affected by noise in the images that accordingly affects the precision of t and R, and so the variability of each voxel point set will in turn affect the overall variability of each matrix equation set for each point. The composite variability between the fixed voxel point set p and a corresponding moving voxel point set q is defined to have a cross-covariance matrix Cpq, more fully described in equation E8 as:
where, n is the number of points in each point set and
Equation E9 gives the SVD value of the cross-covariance Cpq:
Cpq=UDVt E9
where D is a 3×3 diagonal matrix and U and V are orthogonal 3×3 matrices
Equation E10 further defines the rotational R description of the transformation T in terms of U and V orthogonal 3×3 matrices as:
R=UVT E10
Equation E11 further defines the translation transform t description of the transformation T in terms of
t=
Equations E8 through E11 present a method to determine the rigid transformation between two point sets p and q—this process corresponds to step 1152 in
The steps of the registration algorithm are applied iteratively until convergence. The iterative sequence includes a Find Closest Points on Fixed Image 1148 step, a Determine New Transform 1152 step, a Calculate Distances 1156 step, and Converged decision 1160 step.
In the Find Closest Points on Fixed Image 1148 step, corresponding q points are found for each point in the fixed set p. Correspondence is defined by determining the closest edge point on q to the edge point of p. The distance transform image helps locate these closest points. Once p and closest −q pixels are identified, the Determine New Transform 1152 step calculates the rotation R via SVD analysis using equations E8-E10 and translation transform t via equation E11. If, at decision step 1160, the change in the average closest point distance between two iterations is less than 5%, then the predicted-q pixel candidates are considered converged and suitable for receiving the transforms R and t to rigidly register the moving image Transform 1136 onto the common voxels p of the 3D Scan Converted 1108 image. At this point, the rigid registration process is complete as closest proximity between voxel or pixel sets has occurred between the fixed and moving images, and the process continues with fusion at step 1016.
If, however, there is >5% change between the predicted-q pixels and p pixels, another iteration cycle is applied via the Apply Transform 1140 to the Find Closest Points on Fixed Image 1148 step, and is cycled through the converged 1160 decision block. Usually in 3 cycles, though as many as 20 iterative cycles, are engaged until is the transformation T is considered converged.
A representative example for the application of the preferred embodiment for the registration and fusion of a moving image onto a fixed image is shown in
The registration and fusing of common pixel sets p and q from scanplanes having approximately the same φ and rotation θ angles can be repeated for other scanplanes in each 3D data set taken at the first (fixed) and second (moving) anatomical sites. For example, if the composite image 1200C above was for scanplane #1, then the process may be repeated for the remaining scanplanes #2-24 or #2-48 or greater as needed to capture a completed uterine mosaic image. Thus an array similar to the 3D array 240 from
If a third and a fourth 3D data sets are taken, the respective registration, fusing, and assembling into scanplane arrays of composited images is undertaken with the same procedures. In this case, the scanplane composite array similar to the 3D array 240 is composed of a greater mosaic number of registered and fused scanplane images.
A representative example the fusing of two moving images onto a fixed image is shown in
A fourth image similarly could be made to bring about a 4-image mosaic from scanplanes from a fourth 3D data set acquired from the transceiver 10 taking measurements at a fourth anatomical site where the fourth 3D data set is acquired with approximately the same tilt φ and rotation θ angles.
The transceiver 10 is moved to different anatomical sites to collect 3D data sets by hand placement by an operator. Such hand placement could create the acquiring of 3D data sets under conditions in which the tilt φ and rotation θ angles are not approximately equal, but differ enough to cause some measurement error requiring correction to use the rigid registration 1012 algorithm. In the event where the 3D data sets between anatomical sites, either between a moving supine site in relation to its beginning fixed site, or between a moving lateral site with its beginning fixed site, cannot be acquired with the tilt φ and rotation θ angles being approximately the same, then the built-in accelerometer measures the changes in tilt φ and rotation θ angles and compensates accordingly so that acquired moving images are presented if though they were acquired under approximately equal tilt φ and rotation θ angle conditions.
To repeat the scan, the top button of the scanner 10 is repetitively depressed, so that it returns the scan to “0 of 6,” to permit a user to repeat all six scans again. Finally, the scanner 10 is returned to the cradle to upload the raw ultrasound data to computer, intranet, or Internet as depicted in
As with the quadrant and the four in-line scancone measuring methods described earlier, the six-segment procedure ensures that the measurement process detects all amniotic fluid regions. The transceiver 10 projects outgoing ultrasound signals, in this case into the uterine region of a patient, at six anatomical locations, and receives incoming echoes reflected back from the regions of interest to the transceiver 10 positioned at a given anatomical location. An array of scanplane images are obtained for each anatomical location based upon the incoming echo signals. Image enhanced and segmented regions for the scanplane images are determined for each scanplane array, which may be a rotational, wedge, or translationally configured scanplane array. The segmented regions are used to align or register the different scancones into one common coordinate system. Thereafter, the registered datasets are merged with each other so that the total amniotic fluid volume is computed from the resulting fused image.
The blurring and deblurring is achieved by a combination of heat and shock filters. The inputted pixel related data from process 1010A2 is first subjected to a heat filter process block 1010A4. The heat filter block 1010A4 is a Laplacian-based filtering and results in reduction of the speckle noise and smooths or otherwise blurs the edges in the image. The heat filter block 1010A4 is modified via a user-determined stored data block 1010A6 wherein the number of heat filter iterations and step sizes are defined by the user and are applied to the inputted data 1010A2 in the heat filter process block 1010A4. The effect of heat iteration number in progressively blurring and removing speckle from an original image as the number of iteration cycles is increased is shown in
The Intensity-based Segmentation relies on the observation that amniotic fluid is usually darker than the rest of the image. Pixels associated with fluids are classified based upon a threshold intensity level. Thus pixels below this intensity threshold level are interpreted as fluid, and pixels above this intensity threshold are interpreted as solid or non-fluid tissues. However, pixel values within a dataset can vary widely, so a means to automatically determine a threshold level within a given dataset is required in order to distinguish between fluid and non-fluid pixels. The intensity-based segmentation is divided into three steps. A first step includes estimating the fetal body and shadow regions, a second step includes determining an automatic thresholding for the fluid region after removing the body region, and a third step includes removing the shadow and fetal body regions from the potential fluid regions.
The Intensity-Based Segmentation Group includes a fetal body region block 1010B2, wherein an estimate of the fetal shadow and body regions is obtained. Generally, the fetal body regions in ultrasound images appear bright and are relatively easily detected. Commonly, anterior bright regions typically correspond with the dome reverberation of the transceiver 10, and the darker appearing uterus is easily discerned against the bright pixel regions formed by the more echogenic fetal body that commonly appears posterior to the amniotic fluid region. In fetal body region block 1010B2, the fetal body and shadow is found in scanlines that extend between the bright dome reverberation region and the posterior bright-appearing fetal body. A magnitude of the estimate of fetal and body region is then modified by a user-determined input parameter stored in a body threshold data block 1010B4, and a pixel value is chosen by the user. For example, a pixel value of 40 may be selected by the user. An example of the image obtained from blocks 1010B2-4 is panel (c) of
Referring now to the second pathway or the Edge-Based Segmentation Group, the procedural blocks find pixel points on an image having high spatial gradient magnitudes. The edge-based segmentation process begins processing the shock filtered 1010A8 pixel data via a spatial gradients block 1010C2 in which the gradient magnitude of a given pixel neighborhood within the image is determined. The gradient magnitude is determined by the taking the X and Y derivatives using the difference kernels shown in
∥∇I∥=√{square root over (Ix2+Iy2)}
I
x
=I*K
x
I
y
=I*K
y E7
where * is the convolution operator.
Once the gradient magnitude is determined, pixel edge points are determined by a hysteresis threshold of gradients process block 1010C4. In block 1010C4, a lower and upper threshold value is selected. The image is then thresholded using the lower value and a connected component labeling is carried out on the resulting image. The pixel value of each connected component is measured to determine which pixel edge points have gradient magnitude pixel values equal to or greater than the upper threshold value. Those pixel edge points having gradient magnitude pixel values equal to or exceeding the upper threshold are retained. This retention of pixels having strong gradient values serves to retain selected long connected edges which have one or more high gradient points.
Thereafter, the image is thresholded using the upper value, and a connected component labeling is carried out on the resulting image. The hysteresis threshold 1010C4 is modified by a user-determined edge threshold block 1010C6. An example of an application of the second pathway will be shown in panels (b) for the spatial gradients block 1010C2 and (c) for the threshold of gradients process block 1010C4 of
Referring again to
The segmentation resulting from the combination of region and edge information occasionally includes extraneous regions or even holes. A cleanup stage helps ensure consistency of segmented regions in a single scanplane and between scanplanes. The cleanup stage uses morphological operators (such as erosion, dilation, opening, closing) using the Markov Random Fields (MRFs) as disclosed in Forbes et al. (Florence Forbes and Adrian E. Raftery, “Bayesian morphology: Fast Unsupervised Bayesian Image Analysis,” Journal of the American Statistical Association, June 1999, herein incorporated by reference). The combined segmentation images receive the MRFs by being subjected to an In-plane Closing and Opening process block 1010D4. The In-plane opening-closing block 1010D4 block is a morphological operator wherein pixel regions are opened to remove pixel outliers from the segmented region, or that fills in or “closes” gaps and holes in the segmented region within a given scanplane. Block 1010D4 uses a one-dimensional structuring element extending through five scanlines. The closing-opening block is affected by a user-determined width, height, and depth parameter block 1010D6. Thereafter, an Out-of-plane Closing and Opening processing block 1010D8 is applied. The block 1010D8 applies a set of out-of-plane morphological closings and openings using a one-dimensional structuring element extending through three scanlines. Pixel inconsistencies are accordingly removed between the scanplanes. Panel (g) of
The steps of the rigid registration algorithm 1014 correct any overlaps between adjacent 3D scan cones acquired in the 6-section supine grid procedure. The rigid algorithm 1014 first converts the fixed image 1104A2 from polar coordinate terms to Cartesian coordinate terms using the 3D Scan Convert 1014A4 algorithm. Separately, the moving image 1014B2 is also converted to Cartesian coordinates using the 3D Scan Convert 1014B4 algorithm. Next, the edges of the amniotic fluid regions on the fixed and moving images are determined and converted into point sets p and q, respectively by a 3D edge detection process 1014A6 and 1014B6. Also, the fixed image point set, p, undergoes a 3D distance transform process 1014B8 which maps every voxel in a 3D image to a number representing the distance to the closest edge point in p. Pre-computing this distance transform makes subsequent distance calculations and closest point determinations very efficient.
Next, the known initial transform 1014B10, for example, (6, 0, 0) for the Cartesian Tx, Ty, Tz terms and (0, 0, 0) for the θx, θy, θz Euler angle terms, for an inter-transceiver interval of 6 cm, is subsequently applied to the moving image by the transform edges 1014B8 block. This transformed image is then subjected to the Find Closest Points on Fixed Image block 1014C2, similar in operation to the block 1148 of
The RigidRegistration block 1014 typically converges in less than 20 iterations. After applying the initial transformation, the entire registration process is carried out in case there are any overlapping segmented regions between any two images. Similar to the process described in connection with
Similarly, ultrasound signal plot (b) depicts the effects of applying a shock filter to a noisy (speckle rich) signal line (sinuous long dash line) that has been smooth or blurred by the heat filter (short dashed line with sigmoidal appearance). In operation the shock filter results in a generally deblurring or sharpening of the edges of the image that were previously blurred. Adjacent with, but not entirely overlapping with the original signal (solid line) throughout the pixel plot range, the shock filtered plot substantially overlaps the vertical portion of the original signal, but is stay elevated in the low and high pixel ranges. Like in (a), a more abrupt or steep stepped signal plot after shock filtering is obtained without significant removal of speckle. Dependent on the gradient threshold, step size, and iteration number imposed by block 1010A10 upon shock block 1010A8, different overlapping levels of the shock filtered line to that of the original is obtained.
Demonstrations of the algorithmic manipulation of pixels of the present invention are provided in Appendix 1: Examples of Algorithmic Steps. Source code of the algorithms of the present invention is provided in Appendix 2: Matlab Source Code.
While the preferred embodiment of the invention has been illustrated and described, as noted above, many changes can be made without departing from the spirit and scope of the invention. For example, other uses of the invention include determining the areas and volumes of the prostate, heart, bladder, and other organs and body regions of clinical interest. Accordingly, the scope of the invention is not limited by the disclosure of the preferred embodiment.
Systems, methods, and devices for image clarity of ultrasound-based images are described and illustrated in the following figures. The clarity of ultrasound imaging requires the efficient coordination of ultrasound transfer or communication to and from an examined subject, image acquisition from the communicated ultrasound, and microprocessor based image processing. Oftentimes the examined subject moves while image acquisition occurs, the ultrasound transducer moves, and/or movement occurs within the scanned region of interest that requires refinements as described below to secure clear images.
The ultrasound transceivers or DCD devices developed by Diagnostic Ultrasound are capable of collecting in vivo three-dimensional (3-D) cone-shaped ultrasound images of a patient. Based on these 3-D ultrasound images, various applications have been developed such as bladder volume and mass estimation.
During the data collection process initiated by DCD, a pulsed ultrasound field is transmitted into the body, and the back-scattered “echoes” are detected as a one-dimensional (1-D) voltage trace, which is also referred to as a RF line. After envelope detection, a set of 1-D data samples is interpolated to form a two-dimensional (2-D) or 3-D ultrasound image.
A directional indicator panel 22 includes a plurality of arrows that may be illuminated for initial targeting and guiding a user to access the targeting of an organ or structure within an ROI. In particular embodiments if the organ or structure is centered from placement of the transceiver 10A acoustically placed against the dermal surface at a first location of the subject, the directional arrows may be not illuminated. If the organ is off-center, an arrow or set of arrows may be illuminated to direct the user to reposition the transceiver 10A acoustically at a second or subsequent dermal location of the subject. The acrostic coupling may be achieved by liquid sonic gel applied to the skin of the patient or by sonic gel pads to which the transceiver dome 20 is placed against. The directional indicator panel 22 may be presented on the display 54 of computer 52 in harmonic imaging subsystems described in
Transceiver 10A includes an inertial reference unit that includes an accelerometer 22 and/or gyroscope 23 positioned preferably within or adjacent to housing 18. The accelerometer 22 may be operable to sense an acceleration of the transceiver 10A, preferably relative to a coordinate system, while the gyroscope 23 may be operable to sense an angular velocity of the transceiver 10A relative to the same or another coordinate system. Accordingly, the gyroscope 23 may be of conventional configuration that employs dynamic elements, or it may be an optoelectronic device, such as the known optical ring gyroscope. In one embodiment, the accelerometer 22 and the gyroscope 23 may include a commonly packaged and/or solid-state device. One suitable commonly packaged device may be the MT6 miniature inertial measurement unit, available from Omni Instruments, Incorporated, although other suitable alternatives exist. In other embodiments, the accelerometer 22 and/or the gyroscope 23 may include commonly packaged micro-electromechanical system (MEMS) devices, which are commercially available from MEMSense, Incorporated. As described in greater detail below, the accelerometer 22 and the gyroscope 23 cooperatively permit the determination of positional and/or angular changes relative to a known position that is proximate to an anatomical region of interest in the patient. Other configurations related to the accelerometer 22 and gyroscope 23 concerning transceivers 10A,B equipped with inertial reference units and the operations thereto may be obtained from copending U.S. patent application Ser. No. 11/222,360 filed Sep. 8, 2005, herein incorporated by reference.
The transceiver 10A includes (or if capable at being in signal communication with) a display 24 operable to view processed results from an ultrasound scan, and/or to allow an operational interaction between the user and the transceiver 10A. For example, the display 24 may be configured to display alphanumeric data that indicates a proper and/or an optimal position of the transceiver 10A relative to the selected anatomical portion. Display 24 may be used to view two- or three-dimensional images of the selected anatomical region. Accordingly, the display 24 may be a liquid crystal display (LCD), a light emitting diode (LED) display, a cathode ray tube (CRT) display, or other suitable display devices operable to present alphanumeric data and/or graphical images to a user.
Still referring to
To scan a selected anatomical portion of a patient, the transceiver dome 20 of the transceiver 10A may be positioned against a surface portion of a patient that is proximate to the anatomical portion to be scanned. The user actuates the transceiver 10A by depressing the trigger 14. In response, the transceiver 10 transmits ultrasound signals into the body, and receives corresponding return echo signals that may be at least partially processed by the transceiver 10A to generate an ultrasound image of the selected anatomical portion. In a particular embodiment, the transceiver 10A transmits ultrasound signals in a range that extends from approximately about two megahertz (MHz) to approximately about ten MHz.
In one embodiment, the transceiver 10A may be operably coupled to an ultrasound system that may be configured to generate ultrasound energy at a predetermined frequency and/or pulse repetition rate and to transfer the ultrasound energy to the transceiver 10A. The system also includes a processor that may be configured to process reflected ultrasound energy that is received by the transceiver 10A to produce an image of the scanned anatomical region. Accordingly, the system generally includes a viewing device, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display device, or other similar display devices, that may be used to view the generated image. The system may also include one or more peripheral devices that cooperatively assist the processor to control the operation of the transceiver 10A, such a keyboard, a pointing device, or other similar devices. In still another particular embodiment, the transceiver 10A may be a self-contained device that includes a microprocessor positioned within the housing 18 and software associated with the microprocessor to operably control the transceiver 10A, and to process the reflected ultrasound energy to generate the ultrasound image. Accordingly, the display 24 may be used to display the generated image and/or to view other information associated with the operation of the transceiver 10A. For example, the information may include alphanumeric data that indicates a preferred position of the transceiver 10A prior to performing a series of scans. In yet another particular embodiment, the transceiver 10A may be operably coupled to a general-purpose computer, such as a laptop or a desktop computer that includes software that at least partially controls the operation of the transceiver 10A, and also includes software to process information transferred from the transceiver 10A, so that an image of the scanned anatomical region may be generated. The transceiver 10A may also be optionally equipped with electrical contacts to make communication with receiving cradles 50 as discussed in
Referring still to
As described above, the angular movement of the transducer may be mechanically effected and/or it may be electronically or otherwise generated. In either case, the number of lines 48 and the length of the lines may vary, so that the tilt angle φ sweeps through angles approximately between −60° and +60° for a total arc of approximately 120°. In one particular embodiment, the transceiver 10 may be configured to generate approximately about seventy-seven scan lines between the first limiting scan line 44 and a second limiting scan line 46. In another particular embodiment, each of the scan lines has a length of approximately about 18 to 20 centimeters (cm). The angular separation between adjacent scan lines 48 (
The locations of the internal and peripheral scan lines may be further defined by an angular spacing from the center scan line 34B and between internal and peripheral scan lines. The angular spacing between scan line 34B and peripheral or internal scan lines may be designated by angle Φ and angular spacings between internal or peripheral scan lines may be designated by angle Ø. The angles Φ1, Φ2, and Φ3 respectively define the angular spacings from scan line 34B to scan lines 34A, 34C, and 31D. Similarly, angles Ø1, Ø2, and Ø3 respectively define the angular spacings between scan line 31B and 31C, 31C and 34A, and 31D and 31E.
With continued reference to
Whether receiving echogenic signals from non-moving targets within the ROI from processing block 200, or moving targets within the ROI from process block 300, algorithm 120 continues with processing blocks 400A or 400B. Processing blocks 400A and 400B process echogenic datasets of the echogenic signals from process blocks 200 and 300 using a point spread function algorithms to compensate or otherwise suppress motion induced reverberations within the ROI echogenic data sets. Processing block 400A employs nonparametric analysis, and processing block 400B employs parametric analysis and described in
Referring to sub-algorithm 400B, parametric analysis employs an implementation of the CLEAN algorithm that is not iterative. Sub-algorithm 400B comprise comprises an RF line processing block 400B-2, a parametric pulse estimation block 400B-4, a CLEAN algorithm block 400B-6, a CLEAN iteration block 400B-8, and a Scan Convert processing block 400B-10. The point spread function of the transducer is estimated once and becomes a priori information used in the CLEAN algorithm. A single estimate of the pulse is applied to all RF lines in a scan plane and the CLEAN algorithm is applied once to each line. The signal output is then converted for presentation as part of a scan plane image at process block 400B-10. Sub-algorithm 400B is then completed and exits to sub-algorithms 500.
Here u in the heat filter represents the image being processed. The image u is 2D, and is comprised of an array of pixels arranged in rows along the x-axis, and an array of pixels arranged in columns along the y-axis. The pixel intensity of each pixel in the image u has an initial input image pixel intensity (I) defined as u0=I. The value of I depends on the application, and commonly occurs within ranges consistent with the application. For example, I can be as low as 0 to 1, or occupy middle ranges between 0 to 127 or 0 to 512. Similarly, I may have values occupying higher ranges of 0 to 1024 and 0 to 4096, or greater. For the shock filter u represents the image being processed whose initial value is the input image pixel intensity (I): u0=I where the l(u) term is the Laplacian of the image u, F is a function of the Laplacian, and ∥∇u∥ is the 2D gradient magnitude of image intensity defined by equation E3:
∥∇u∥=√{square root over (ux2+uy2)} E3:
Where u2x=the square of the partial derivative of the pixel intensity (u) along the x-axis, u2y=the square of the partial derivative of the pixel intensity (u) along the y-axis, the Laplacian l(u) of the image, u, is expressed in equation E4:
l(u)=uxxux2+2uxyuxuy+uyyuy2
Equation E9 relates to equation E6 as follows:
ux is the first partial derivative
of u along the x-axis,
ux uy is the first partial derivative
of u along the y-axis,
ux ux2 is the square of the first partial derivative
of u along the x-axis,
ux uy2 is the square of the first partial derivative
of u along the y-axis,
ux uxx is the second partial derivative
of u along the x-axis,
ux uyy is the second partial derivative
of u along the y-axis,
uxy is cross multiple first partial derivative
of u along the x and y axes, and
uxy the sign of the function F modifies the Laplacian by the image gradient values selected to avoid placing spurious edges at points with small gradient values:
where t is a threshold on the pixel gradient value ∥∇u∥.
The combination of heat filtering and shock filtering produces an enhanced image ready to undergo the intensity-based and edge-based segmentation algorithms as discussed below. The enhanced 3D data sets are then subjected to a parallel process of intensity-based segmentation at process block 510 and edge-based segmentation at process block 512. The intensity-based segmentation step uses a “k-means” intensity clustering technique where the enhanced image is subjected to a categorizing “k-means” clustering algorithm. The “k-means” algorithm categorizes pixel intensities into white, gray, and black pixel groups. Given the number of desired clusters or groups of intensities (k), the k-means algorithm is an iterative algorithm comprising four steps: Initially determine or categorize cluster boundaries by defining a minimum and a maximum pixel intensity value for every white, gray, or black pixels into groups or k-clusters that are equally spaced in the entire intensity range. Assign each pixel to one of the white, gray or black k-clusters based on the currently set cluster boundaries. Calculate a mean intensity for each pixel intensity k-cluster or group based on the current assignment of pixels into the different k-clusters. The calculated mean intensity is defined as a cluster center. Thereafter, new cluster boundaries are determined as mid points between cluster centers. The fourth and final step of intensity-based segmentation determines if the cluster boundaries significantly change locations from their previous values. Should the cluster boundaries change significantly from their previous values, iterate back to step 2, until the cluster centers do not change significantly between iterations. Visually, the clustering process is manifest by the segmented image and repeated iterations continue until the segmented image does not change between the iterations.
The pixels in the cluster having the lowest intensity value—the darkest cluster—are defined as pixels associated with internal cavity regions of bladders. For the 2D algorithm, each image is clustered independently of the neighboring images. For the 3D algorithm, the entire volume is clustered together. To make this step faster, pixels are sampled at 2 or any multiple sampling rate factors before determining the cluster boundaries. The cluster boundaries determined from the down-sampled data are then applied to the entire data.
The edge-based segmentation process block 512 uses a sequence of four sub-algorithms. The sequence includes a spatial gradients algorithm, a hysteresis threshold algorithm, a Region-of-Interest (ROI) algorithm, and a matching edges filter algorithm. The spatial gradient algorithm computes the x-directional and y-directional spatial gradients of the enhanced image. The hysteresis threshold algorithm detects salient edges. Once the edges are detected, the regions defined by the edges are selected by a user employing the ROI algorithm to select regions-of-interest deemed relevant for analysis.
Since the enhanced image has very sharp transitions, the edge points can be easily determined by taking x- and y-derivatives using backward differences along x- and y-directions. The pixel gradient magnitude ∥∇I∥ is then computed from the x- and y-derivative image in equation E5 as:
∥∇I∥=√{square root over (Ix2+Iy2)}
Where I2x=the square of x-derivative of intensity and I2y=the square of y-derivative of intensity along the y-axis.
Significant edge points are then determined by thresholding the gradient magnitudes using a hysteresis thresholding operation. Other thresholding methods could also be used. In hysteresis thresholding, two threshold values, a lower threshold and a higher threshold, are used. First, the image is thresholded at the lower threshold value and a connected component labeling is carried out on the resulting image. Next, each connected edge component is preserved which has at least one edge pixel having a gradient magnitude greater than the upper threshold. This kind of thresholding scheme is good at retaining long connected edges that have one or more high gradient points.
In the preferred embodiment, the two thresholds are automatically estimated. The upper gradient threshold is estimated at a value such that at most 97% of the image pixels are marked as non-edges. The lower threshold is set at 50% of the value of the upper threshold. These percentages could be different in different implementations. Next, edge points that lie within a desired region-of-interest are selected. This region of interest algorithm excludes points lying at the image boundaries and points lying too close to or too far from the transceivers 10A,B. Finally, the matching edge filter is applied to remove outlier edge points and fill in the area between the matching edge points.
The edge-matching algorithm is applied to establish valid boundary edges and remove spurious edges while filling the regions between boundary edges. Edge points on an image have a directional component indicating the direction of the gradient. Pixels in scanlines crossing a boundary edge location can exhibit two gradient transitions depending on the pixel intensity directionality. Each gradient transition is given a positive or negative value depending on the pixel intensity directionality. For example, if the scanline approaches an echo reflective bright wall from a darker region, then an ascending transition is established as the pixel intensity gradient increases to a maximum value, i.e., as the transition ascends from a dark region to a bright region. The ascending transition is given a positive numerical value. Similarly, as the scanline recedes from the echo reflective wall, a descending transition is established as the pixel intensity gradient decreases to or approaches a minimum value. The descending transition is given a negative numerical value.
Valid boundary edges are those that exhibit ascending and descending pixel intensity gradients, or equivalently, exhibit paired or matched positive and negative numerical values. The valid boundary edges are retained in the image. Spurious or invalid boundary edges do not exhibit paired ascending-descending pixel intensity gradients, i.e., do not exhibit paired or matched positive and negative numerical values. The spurious boundary edges are removed from the image.
For bladder cavity volumes, most edge points for blood fluid surround a dark, closed region, with directions pointing inwards towards the center of the region. Thus, for a convex-shaped region, the direction of a gradient for any edge point, the edge point having a gradient direction approximately opposite to the current point represents the matching edge point. Those edge points exhibiting an assigned positive and negative value are kept as valid edge points on the image because the negative value is paired with its positive value counterpart. Similarly, those edge point candidates having unmatched values, i.e., those edge point candidates not having a negative-positive value pair, are deemed not to be true or valid edge points and are discarded from the image.
The matching edge point algorithm delineates edge points not lying on the boundary for removal from the desired dark regions. Thereafter, the region between any two matching edge points is filled in with non-zero pixels to establish edge-based segmentation. In a preferred embodiment of the invention, only edge points whose directions are primarily oriented co-linearly with the scanline are sought to permit the detection of matching front wall and back wall pairs of a bladder cavity, for example the left or right ventricle.
Referring again to
After combining the segmentation results, the combined pixel information in the 3D data sets In a fifth process is cleaned at process block 516 to make the output image smooth and to remove extraneous structures not relevant to bladder cavities. Cleanup 516 includes filling gaps with pixels and removing pixel groups unlikely to be related to the ROI undergoing study, for example pixel groups unrelated to bladder cavity structures. Sub-algorithm 500 is then completed and exits to sub-algorithm 600.
An embodiment related to cannula insertion generally includes an ultrasound probe attached to a first camera and a second camera and a processing and display generating system that is in signal communication with the ultrasound probe, the first camera, and/or the second camera. A user of the system scans tissue containing a target vein using the ultrasound probe and a cross-sectional image of the target vein is displayed. The first camera records a first image of a cannula in a first direction and the second camera records a second image of the cannula in a second direction orthogonal to the first direction. The first and/or the second images are processed by the processing and display generating system along with the relative positions of the ultrasound probe, the first camera, and/or the second camera to determine the trajectory of the cannula. A representation of the determined trajectory of the cannula is then displayed on the ultrasound image.
First, a user employs the ultrasound probe 1010 and the processing and display generating system 1061 to generate a cross-sectional image of a patient's arm tissue containing a vein to be cannulated (“target vein”) 1019. This could be done by one of the methods disclosed in the related patents and/or patent applications which are herein incorporated by reference, for example. The user then identifies the target vein 1019 in the image using methods such as simple compression which differentiates between arteries and/or veins by using the fact that veins collapse easily while arteries do not. After the user has identified the target vein 1019, the ultrasound probe 1010 is affixed to the patient's arm over the previously identified target vein 19 using a magnetic tape material 1012. The ultrasound probe 1010 and the processing and display generating system 1061 continue to generate a 2D cross-sectional image of the tissue containing the target vein 1019. Images from the cameras 1014, 1018 are provided to the processing and display generating system 1061 as the cannula 1020 is approaching and/or entering the arm of the patient.
The processing and display generating system 1061 locates the cannula 1020 in the images provided by the cameras 1014, 1018 and determines the projected location at which the cannula 1020 will penetrate the cross-sectional ultrasound image being displayed. The trajectory of the cannula 1020 is determined in some embodiments by using image processing to identify bright spots corresponding to micro reflectors previously machined into the shaft of the cannula 1020 or a needle used alone or in combination with the cannula 1020. Image processing uses the bright spots to determine the angles of the cannula 1020 relative to the cameras 1014, 1018 and then generates a projected trajectory by using the determined angles and/or the known positions of the cameras 1014, 1018 in relation to the ultrasound probe 10. In other embodiments, determination of the cannula 1020 trajectory is performed using edge-detection algorithms in combination with the known positions of the cameras 1014, 1018 in relation to the ultrasound probe 1010, for example.
The projected location may be indicated on the displayed image as a computer-generated cross-hair 1066, the intersection of which is where the cannula 1020 is projected to penetrate the image. When the cannula 1020 does penetrate the cross-sectional plane of the scan produced by the ultrasound probe 1010, the ultrasound image confirms that the cannula 1020 penetrated at the location of the cross-hair 1066. This gives the user a real-time ultrasound image of the target vein 1019 with an overlaid real-time computer-generated image of the position in the ultrasound image that the cannula 1020 will penetrate. This allows the user to adjust the location and/or angle of the cannula 1020 before and/or during insertion to increase the likelihood they will penetrate the target vein 1019. Risks of pneumothorax and other adverse outcomes should be substantially reduced since a user will be able to use normal “free” insertion procedures but have the added knowledge of knowing where the cannula 1020 trajectory will lead.
The processing and display generating system 1061 is composed of a display 1064 and a block 1062 containing a computer, a digital signal processor (DSP), and analog to digital (A/D) converters. As discussed for
While the preferred embodiment of the invention has been illustrated and described, as noted above, many changes can be made without departing from the spirit and scope of the invention. For example, a three dimensional ultrasound system could be used rather than a 2D system. In addition, different numbers of cameras could be used along with image processing that determines the cannula 1020 trajectory based on the number of cameras used. The two cameras 1014, 1018 could also be placed in a non-orthogonal relationship so long as the image processing was adjusted to properly determine the orientation and/or projected trajectory of the cannula 1020. Also, an embodiment of the invention could be used for needles and/or other devices which are to be inserted in the body of a patient. Additionally, an embodiment of the invention could be used in places other than arm veins. Regions of the patient's body other than an arm could be used and/or biological structures other than veins may be the focus of interest. As regards ultrasound-based algorithms, alternate embodiments may be configured to image acquisitions other than ultrasound, for example X-ray, visible and infrared light acquired images. Accordingly, the scope of the invention is not limited by the disclosure of the preferred embodiment.
The following applications are incorporated by reference as if fully set forth herein: U.S. application Ser. No. 11/119,355 filed Apr. 29, 2005; Ser. No. 11/362,368 filed Feb. 26, 2006; Ser. No. 11/680,380 filed Feb. 28, 2007 and Ser. No. 11/925,654 filed Oct. 26, 2007.
Number | Date | Country | |
---|---|---|---|
60566823 | Apr 2004 | US | |
60423881 | Nov 2002 | US | |
60400624 | Aug 2002 | US | |
60423881 | Nov 2002 | US | |
60423881 | Nov 2002 | US | |
60400624 | Aug 2002 | US | |
60470525 | May 2003 | US | |
60760677 | Jan 2006 | US | |
60633485 | Dec 2004 | US | |
60566823 | Apr 2004 | US | |
60423881 | Nov 2002 | US | |
60400624 | Aug 2002 | US | |
60423881 | Nov 2002 | US | |
60423881 | Nov 2002 | US | |
60400624 | Aug 2002 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 11119355 | Apr 2005 | US |
Child | 11925887 | US | |
Parent | 10165556 | Jun 2002 | US |
Child | PCT/US03/14785 | US | |
Parent | 10165556 | Jun 2002 | US |
Child | 10633186 | US | |
Parent | 11362368 | Feb 2006 | US |
Child | 10165556 | US | |
Parent | PCT/US05/43836 | Dec 2005 | US |
Child | 11362368 | US | |
Parent | 11295043 | Dec 2005 | US |
Child | PCT/US05/43836 | US | |
Parent | PCT/US05/30799 | Aug 2005 | US |
Child | 11362368 | US | |
Parent | PCT/US05/31755 | Sep 2005 | US |
Child | PCT/US05/30799 | US | |
Parent | 11119355 | Apr 2005 | US |
Child | PCT/US05/31755 | US | |
Parent | 10165556 | Jun 2002 | US |
Child | PCT/US03/14785 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 10701955 | Nov 2003 | US |
Child | 11119355 | US | |
Parent | 10443126 | May 2003 | US |
Child | 10701955 | US | |
Parent | PCT/US03/24368 | Aug 2003 | US |
Child | 10443126 | US | |
Parent | PCT/US03/14785 | May 2003 | US |
Child | 11119355 | US | |
Parent | 10633186 | Jul 2003 | US |
Child | 11119355 | US | |
Parent | 10701955 | Nov 2003 | US |
Child | 11119355 | US | |
Parent | 10443126 | May 2003 | US |
Child | 10701955 | US | |
Parent | PCT/US03/24368 | Aug 2003 | US |
Child | 11119355 | US | |
Parent | PCT/US03/14785 | May 2003 | US |
Child | 11119355 | US | |
Parent | 10633186 | Jul 2003 | US |
Child | 11119355 | US |