The present invention relates to the determination of mechanical properties of material, particularly of materials such as human or animal tissue, and the biomechanical properties of such tissue useful in clinical diagnosis. It also relates to improvements in the display of position-dependent properties of materials, especially in the visualisation of such properties.
A large variety of techniques are known for measuring various mechanical properties of materials, for example in the field of non-destructive testing and also in the clinical field as an aid to clinical diagnosis. In many such techniques the properties of the material which are measured can be displayed as an image (which may be two, three or four dimensional) and such images may be used as a basis for diagnosis along with other quantitatively or qualitatively measured properties. Over the years there have been continuous developments in the measurement of new material properties, and also in techniques for assisting the operator to interpret such information, for example by improvements in “visualisation”, meaning the visual display of those properties.
One such technique which is useful in many fields (including non-destructive testing and human and veterinary clinical practice) is ultrasound imaging. Ultrasound imaging is very well known and ultrasound data is typically displayed as a grey scale image representing a section in the depth direction (axial direction) through the material under examination. In the course of an ultrasound examination an ultrasound probe is brought into contact with the tissue under examination. In the clinical field it is common practice for ultrasonographers to use the probe to distort the tissue by pushing it into the tissue and observe how the tissue structures visible in the ultrasound image move and change. Thus in addition to a static image of the tissue, the reaction of the tissue to motion can be observed. This can assist in determining the nature of structures observed in the image.
It has also been proposed to calculate from a sequence of images of the tissue undergoing deformation the elasticity (or more accurately the axial strain of the tissue). It should be noted that in this specification the term elasticity will be used for the change in length per original length, as it is the term of the art in medical imaging, though to be technically accurate, for example in materials science, this would be referred to as strain. Techniques for calculating elasticity from ultrasound data are disclosed, for example, in the following documents incorporated herein by reference:
Such elasticity information can conveniently be displayed to the operator as a colour overlay on the conventional grey scale ultrasound image with different colours corresponding to different values of elasticity.
However, elasticity does not always resolve ambiguities seen in the basic ultrasound image, and it has also been found that the use of the colour overlay on the conventional grey scale ultrasound image can be distracting for the operator and cause them to miss significant features.
A first aspect of the present invention relates to the quantification of a further mechanical property of the material in an object undergoing examination which is of great use in providing additional information and removing ambiguity between images of different structures which have similar stiffness (e.g. in the clinical field for example fibroadenomas and tumours). In more detail it relates to a method of quantifying the mobility of structures, namely the degree to which one region of material moves independently from another. This is achieved by inducing a movement in the material and differentiating regions in the image which move in a uniform and connected manner from those which move irregularly or independently from neighbouring regions. The quantified measure of mobility is here called “slip”, there being a “slip value” for each of a plurality of parts of the material under examination.
As a first aspect therefore, the invention provides a method of obtaining quantified slip values of material in an object undergoing an imaging examination to provide a sequence of images of the material, comprising physically distorting the material, measuring the resultant motion of the material through the sequence of images to estimate the degree of relative mobility of different areas of the material, and calculating the slip values on the basis of said resultant motion.
The examination may be an ultrasound examination, e.g. B-mode, or may be an MRI or other imaging modality, to provide a primary image from which the slip values may be calculated.
The slip values may be displayed visually superimposed on the primary image of the material, for example with different slip values corresponding to different values of an attribute of an image overlay visually superimposed on the primary image. The image attribute which varies can be the colour or intensity.
In addition, the transparency of the image overlay may be varied depending on the slip value so that the overlay is more transparent where slip values are closer to those expected of normal material. This means that not only does the colour change according to the slip value, but also that the colour overlay is stronger where the slip values are abnormal and more transparent where they are normal, which helps the operator direct attention to regions of interest. Preferably the transparency varies progressively depending on the slip value. The dependency may be user-selectable so that low transparency can be set to correspond to whatever slip values are of most interest. In the field of breast ultrasound examination for breast cancer, for example, benign objects of interest such as cysts and some hard solid lesions tend to have high slip values (high mobility) and so the image overlay is made less transparent for areas of high slip value, thus providing a contrast with hard malignant structures which tend to have low slip values (low mobility).
The slip values are based on the degree of relative mobility which preferably is the non-uniformity of movement or the amount of movement of a structure relative to neighbouring regions. This may be calculated from a motion field which in turn is calculated from a sequence of images taken as the material is deformed. Such a motion field can be calculated using a conventional motion estimation algorithm. Preferably the estimate of the degree of mobility and the slip values are calculated from the quality of the tracking achieved by the motion estimation algorithm. Thus regions where the quality of the tracking is high correspond to low mobility material, whereas regions where the quality of the tracking is low correspond to high mobility material. Preferably the quality of the tracking is taken as the tracking error returned by the motion estimation algorithm. This of course assumes a minimum level of tracking quality is obtained for a sufficient portion of the region of interest. If the quality of the tracking overall is insufficient, it may be more appropriate to indicate that general tracking failure or excessive noise occurred since in this situation the correspondence between tracking quality and tissue mobility will be minimized.
The estimate of the degree of mobility may be calculated as a linear sum or any other function of the estimated tracking errors returned by the motion estimation algorithm, or of the image correlation of the regions tracked by the motion estimation algorithm.
The estimate of the degree of mobility may be calculated from a strain field calculated from a sequence of images taken as the material is deformed, e.g. using an elasticity strain estimation algorithm, and more particularly from the quality of the strain estimation achieved by the strain estimation algorithm or from the strain estimation error returned by the strain estimation algorithm.
The estimate of the degree of mobility may be calculated from a combination of at least two of the estimation qualities achieved by the motion estimation algorithm, the strain estimation algorithm, and image correlation.
The deformation of the material may be applied manually by the operator, or by means of a machine which provides automated incremental displacements of the material to provide contact force externally at the surface. In the case of an ultrasound examination, the force may be applied by using the ultrasound probe itself to distort the material or by utilising acoustic force in the material due to the ultrasound propagation. The deformation of the material may also be achieved by natural sources of motion, such as, in a biological subject: breathing or cardiac pulse.
A second aspect of the invention relates to improving the visualisation of position-dependent properties of a subject. Accordingly, this aspect of the invention provides a method of displaying measurements of a plurality of different position-dependent properties of a subject, comprising displaying a primary image representing a first of said position-dependent properties, and displaying visually superimposed over the primary image an overlay image whose image attributes vary to represent the values of a second of said position-dependent properties, and wherein the transparency of the overlay image spatially varies progressively in dependence on the value of said second of said position-dependent properties.
Thus two position-dependent properties can be displayed, one in the primary image and one as an overlay image which is visually superimposed. To avoid confusion in the image, the overlay image is rendered transparent in certain areas, preferably where the values of the second of the position-dependent properties are of low interest (for example where the values are normal). Thus, preferably, areas of the overlay image representing normal values of the second of the position-dependent properties are given higher transparency than areas representing interesting or abnormal, e.g. clinically relevant, values.
The overlay image may contain areas which are completely transparent, i.e. effectively the values of the second of the position-dependent properties are not shown. But in areas which are not completely transparent, the transparency varies according to the values of the second of the position-dependent properties.
The relationship between the transparency and the values of the second of the position-dependent properties may be user-selectable so that the user can adapt the display to different applications. The attribute of the overlay image which varies to represent the values of the second of the position-dependent properties can be the colour or intensity, though colour is particularly effective when overlaid on a grey scale image such as an ultrasound image.
The second of the position-dependent properties may be a mechanical property, for example elasticity or mobility. The second of the position-dependent properties can be the elasticity of the material, with areas of high stiffness corresponding to low transparency. Alternatively, the second of the position-dependent properties can be slip values representing estimates of the mobility of the material, with areas of high mobility corresponding to low transparency.
It is particularly effective if the original image and composite images consisting of original image with the overlay are displayed alongside each other, preferably with the same scale and area, allowing easy comparison.
Thus a third aspect of the invention provides a method comprising displaying adjacent to each other a first image consisting of a primary image of material under examination, a second image representing the elasticity of the material under examination, and a third image displaying slip values representing estimates of the mobility of different areas of the material under examination.
The primary image can be an ultrasound image, or MRI or other medical imaging modality.
As indicated above, preferably the three images are displayed with the same scale and area, and the second and third images may be two composite images including overlays indicating the values of additional properties of the material such as elasticity (strain) and slip (mobility). Preferably the transparency of the overlays in the composite images is varied as mentioned above.
The images displayed in the three aspects above can be still or video images. As video images, they may be displayed in real-time, or pre-recorded and played back. Further, although the invention is exemplified above and below in the field of ultrasound, the invention is applicable to other imaging modalities.
The images of the object may be in 1, 2 or 3 dimensions and the subject may be any biological, e.g. human, animal or plant, or non-biological, material.
The invention is conveniently embodied in software for processing ultrasound image data and thus the invention extends to a computer program comprising program code means for executing the methods above on a programmed computer. The invention also extends to an ultrasound imaging apparatus including a processor adapted to process ultrasound signals and to display them as set out above.
The invention will be further described by way of non-limitative example with reference to the accompanying drawings in which:
a) to (e) show similar image triples for tissue including different structures of interest.
The elasticity and slip will be visualised by displaying them as coloured image overlays on the original ultrasound image. Thus, in step 130 the overlays are prepared from the elasticity and slip data and in particular the transparencies of the overlays are varied with the clinically relevant regions (high stiffness and high slip) made more opaque (less transparent) and the less relevant regions (low stiffness, low slip) made more transparent. Finally, in step 140 an image triple consisting of the original ultrasound image with two composite images alongside it are displayed, the two composite images being the basic ultrasound image with the elasticity information overlaid and the basic ultrasound image with the slip values overlaid. The image triple gives a more complete picture of the tissue properties and can increase the confidence in, and power of, ultrasound imaging diagnosis as will be illustrated below by some specific examples.
A specific example will now be described with reference to an ultrasound acquisition of four sequential images.
The invention is applicable to conventional methods of ultrasound elasticity imaging as set out, for example, in the publications by J Ophir et al., L Gao et al. or B S Gara et al. mentioned above. Deriving the mobility information (slip values) requires at least two ultrasound images, but preferably more to reduce estimation noise as described in the paper by T Varghese et al. referenced above. Thus in general, an ultrasound acquisition of N sequential images, I1-IN, is made while the tissue is slowly compressed in the axial direction of the ultrasound scan plane (though alternatively the deformation can be a decompression). Motion is then estimated between each consecutive pair of ultrasound images using a conventional motion algorithm such as sum-of-squared differences block-matching as described in the paper by I A Hein and W D O'Brien referenced above. Non-consecutive image pairs can be also used, with the pairs selected to improve the overall quality of the motion estimation. Other motion estimation algorithms are, of course, available and can be used preferably if they have an associated error or quality output.
In this particular algorithm a sum-of-squared-differences error:
E
12(x)=(I2(T12(x)+x)−I1(x))2
is minimized over multiple discrete image regions (blocks) to find the optimal motion estimate:
T
12(x)=argmin(E12(x)),
where x is the spatial coordinate and can be in multiple dimensions.
This results in N−1 motion field estimates, denoted T12 through T(N−1) N, and N−1 error estimates, denoted E12 through E(N−1)N, between the successive image pairs.
If N>2, the multiple motion and error estimates are combined into single estimates:
T=T
12
+T
23(T12+x)+T34(T23(T12+x)+T12+x)+ . . . ,
E=E
12
+E
23(T12+x)+E34(T23(T12+x)+T12+x)+ . . . .
The compositions can be obtained by interpolating each motion and error field at the locations specified by the previous motion fields. Additionally, the error fields can be combined using any function instead of the linear summation shown.
The composed motion field is used to derive an elasticity estimate according to a conventional axial strain estimation algorithm such as a least-squares gradient estimator.
The composed error field is used as the estimate of mobility (slip values). Alternatively, if estimate quality, Q, is available instead of an estimate error, it can be used by subtracting it from its maximum value:
E
ij=max(Q)−Qij
In this manner, the quality or similarity metric can be converted and then composed like the error metric to be used as an estimate of mobility.
The results are presented in an image triple with the ultrasound B-mode, elasticity, and slip images (step 140). Ultrasound B-mode is a grey scale image and is used as the background for all three images. The elasticity and slip images are created by overlaying the estimated elasticity and mobility data as colour over the ultrasound B-mode image background. In the attached drawings to be published, to assist viewing in monochrome, the different coloured regions have been arbitrarily outlined and shaded so that they can be seen, but in the originally filed drawings and in practice there are no such outlines—just a progressively varying colour and varying transparency overlay.
Depending on the application, different elasticity and mobility values have increased importance. For instance, in breast tissue diagnosis hard tissue often correlates to malignancy, and highly mobile tissue often correlates to benignity. To highlight the features of interest, the harder and more mobile regions are made redder and more opaque, whereas the softer and less mobile regions are made bluer and more transparent. Thus this spatially varying transparency is a function of the elasticity or slip value according to some arbitrary mapping function which can be a simple linear mapping.
The data flow for this example is shown in
Four ultrasound images, shown in
With the four ultrasound images (
The three motion fields (
T=T
12
+T
23(T12+x)+T34(T23(T12+x)+T12+x).
This required T23 to be interpolated at the locations specified by T12, and T34 to be interpolated at the locations specified by T23 and T12. Spline interpolation was used for this task. In a similar manner the composed error field, E, was calculated:
E=E
12
+E
23(T12+x)+E34(T23(T12+x)+T12+x).
This required E23 and E34 to be interpolated like T23 and T34, and spline interpolation was also used for this task. Thus the three motion fields shown in
The elasticity image is obtained by calculating the gradient of the motion image, or how much areas have changed in size (i.e. the strain). Thus using the composed motion field of
Three images were created for the final result shown in
a to e illustrate image triples obtained in accordance with the procedure above for tissues containing different structures of interest.
In the example above, the tracking error obtained from the motion estimation algorithm is used as a measure of slip (or mobility). In general any parameter which is indicative of areas where tracking is not good can be used. Typically low quality tracking is caused by out of plane motion or by movement of areas which are filled with fluid and thus are random or highly spatially-varying. Additionally, any parameter which is indicative of areas of increased mobility, that is areas with motion independent or greater than neighbouring areas, can also be used.
Although in the example above the transparency of the overlays is adjusted to lie between 0.5 (partially transparent) and 1 (fully transparent), the relationship can be different, or can be set differently for different applications. Control of this may be given to the user, who can then try different relationships between transparency and the values to choose one which gives the greatest distinctiveness of the regions of interest. The values above are chosen in order to de-emphasise normal responses from the tissue in order to avoid distracting the user from regions of interest.
Number | Date | Country | Kind |
---|---|---|---|
0712432.4 | Jun 2007 | GB | national |
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/GB2008/002172 | 6/25/2008 | WO | 00 | 2/23/2010 |