The present invention relates to a method for aiding an operator to determine a position of a region of interest in a breast and a mammography apparatus implementing it.
The region of interest corresponds in particular to a portion of the breast from which a sample is to be withdrawn for diagnostic purposes, such as biopsy procedures.
It is known to guide breast biopsy procedures using 3D breast imaging systems. Such a solution is disclosed and shown, for example, in Patent Document EP3023076A1.
More recently, solutions combining both 3D breast imaging procedures of the “tomosynthesis” type and 2D breast imaging procedures have been proposed. Solutions of this kind are disclosed and shown, for example, in Patent Documents EP2491863A1 and EP2656789A1.
The general object of the present invention is to provide an alternative and improved solution with respect to the known ones.
This object is achieved by the method and apparatus having the characteristics explained in the herein appended claims which form an integral part of the present invention.
According to the present invention, both a 3D breast imaging procedure of the “tomosynthesis” type and a 2D breast imaging procedure of the “dual energy” or “DEM” type, in particular and preferably of the “CESM” (=Contrast-Enhanced Spectral Mammography) type, are combined; indeed, advantageously, “dual-energy” mammography, in particular that carried out with a contrast medium, allows particularly effective visualisation for performing biopsies.
“Dual energy mammography” or “DEM” is a well-known technique whereby two images of a breast are taken from the same position in rapid succession (e.g., seconds or less). The two images are obtained by irradiating the breast with two different spectra (in the X-ray range) because they are centred on two different average energies. There is also an alternative known technique whereby the breast is irradiated using the same spectrum and the two images are obtained by performing a spectral separation using a detector capable of discriminating the energy of the photons striking it. Hence, in both cases, each of the two images is associated with a different spectrum (and its relative average energy).
According to preferred embodiments of the present invention, 2D imaging occurs prior to 3D imaging; it is thereby possible, for example, to centre the breast according to DEM or CESM before proceeding with tomosynthesis and subsequently biopsy. If the DEM or CESM possibly indicates that the breast is not well positioned (in particular, if it is established that the biopsy needle cannot reach a certain lesion on the breast, for example because that lesion is not displayed within an appropriate for accessing to/withdrawing from the breast), it can be repositioned by relieving or removing the pressure on the breast by the mammography apparatus “compression plate” or “compressor”. Experts in the field use the term “scout image” to identify an image which is primarily used for visually searching for lesions.
In order to benefit from the advantages of both types of imaging, the present invention proposes to enable markers to be displayed on the 2D image and/or on one or more “sections” of the 3D image, and to ensure a correspondence between the 2D image and the “sections” of the 3D image as well as between the markers on the two types of images.
This is achieved by three-dimensional to two-dimensional mapping and two-dimensional transformation operations that pass through a virtual 2D image created from the 3D image.
The problem of matching a mammographic image with a tomosynthesis image is addressed, for example, in the article by Jan Ehrhardt et al. entitled “Automatic Correspondence Detection in Mammogram and Breast Tomosynthesis Images” (Proceeding of SPIE, vol. 8314, 2012). The article proposes two approaches.
According to the first approach, by means of an iterative process, the tomosynthetic image, which is three-dimensional, is recorded against the mammographic image, which is two-dimensional, through a repeated generation of “DRR” (which are two-dimensional images that can be considered as “virtual”) and repeated optimisation of the similarity between “Mammogram” and “DRR”. According to the second approach, the mammographic image and the central projection of the tomosynthetic image are firstly recorded with each other and then each of the other projections of the tomosynthetic image is recorded with respect to the central one; these recordings are not based on “virtual” images.
These approaches are not suitable for use in guiding a biopsy procedure. In fact, both of them modify the tomosynthetic image that is at the very basis of a guided biopsy procedure. In addition, both of them require heavy and therefore time-consuming calculations. Therefore, these approaches can only be used for displaying purposes, and, moreover, are designed for images obtained at very different times (at least hours, but typically days or weeks or months) and therefore not during the same compression operation.
An advantage of the approach according to the present invention over a possible stereotactic examination performed entirely in a “dual energy” mode is that examination times are shortened. Experts in the field know that the tomosynthesis image-guided biopsy method significantly shortens the procedure times. This approach is therefore extremely advantageous for the comfort of the patient (who is subjected to discomfort and anxiety during breast imaging procedures and possible subsequent biopsy), but even more important in the case of performing imaging procedures with concurrent injection of a contrast medium, which has a limited residence time inside the human body.
An advantage of the approach according to the present invention over a possible tomo-guided biopsy performed entirely in a “dual energy” mode is the significant reduction in the X-ray dose absorbed by the patient being examined; in fact, the X-ray delivery for a “dual energy” tomosynthesis corresponds to twice the delivery for a “single energy” (i.e., conventional) tomosynthesis. In addition, the CEST (=Contrast-Enhanced Spectral Tomosynthesis) technique, or “dual energy” tomosynthesis, involves complications as artefacts typical of CESM and artefacts typical of tomosynthesis combine together, resulting in problems with the visibility of the lesions.
In order to amplify the correlation between the two types of imaging, the energy of 3D imaging can be chosen appropriately and advantageously; this choice can be made according to optimisation criteria that differ from the general criterion of anatomical visibility in conventional mammography; for example, choice criteria can be used that are similar to those for optimising the acquisition with energy further away from the mammographic range in dual energy imaging. If, for example, during a biopsy procedure, the scout image derives from a CESM, it must be assumed that an attempt is being made to take a tissue sample from a contrast-medium highly capturing area. In this case, the tomosynthesis image will therefore preferably be taken at an energy such that there is a photon absorption peak by the contrast medium, neglecting the optimisation of the visibility of the surrounding tissues. This energy is presumably equal to or nearly equal to the highest energy of the pair of energies used for CESM.
A further advantage of the imaging combination according to the present invention is given by the possibility of inserting an anti-scatter grid during scout image acquisition. In normal stereotactic examinations, the standard method of removing scattered radiation, i.e., using an anti-scatter grid, cannot be used because the stereotactic projections are not compatible with the vertical focusing direction of the grid. In order to make the display of the stereotactic images and the (vertical) image used for centring homogeneous, the anti-scatter grid is not used for the “scout” image either. However, this results in a different (degraded) image quality if compared to the mammographic images on the basis of which the patient was probably recommended for further diagnosis with the biopsy. As, according to the present invention, the “scout” image acquisition mode (DEM or CESM) and the image acquisition mode used for pointing (tomosynthesis) are already naturally non-homogeneous, the anti-scatter grid can be advantageously used in the “scout” image acquisition (DEM or CESM).
According to the present invention, it is advantageously possible to perform a strong relation between two images of different types; recording techniques are employed which involve, on the one hand, a composite 2D image deriving from a CESM or one of the two images underlying a CESM and, on the other hand, a virtual 2D image obtained by mapping on a plane a 3D image deriving from a tomosynthesis, and which are advantageous especially if the two different images are acquired in sequence and if they are spaced by a relatively long time distance (e.g. of the order of one minute) during the same compression operation; in fact, in this case, there are discrepancies (more or less evident) in the observed anatomy due to relatively small movements of the patient, e.g. of the order of a millimetre.
The present invention shall become more readily apparent from the detailed description that follows to be considered together with the accompanying drawings in which:
As it can be easily understood, there are various ways of practically implementing the present invention which is defined in its main advantageous aspects in the appended claims and is not limited either to the following detailed description or to the appended claims.
The apparatus 1 of
As it is clear,
In particular and as graphically highlighted, the detector 6 is of electronic type and integrates a two-dimensional array, or matrix, of elementary sensors, for example one per pixel, which defines an acquisition plane 61; in
A first function of the apparatus 1 is therefore to perform breast imaging procedures; in
A second function of the apparatus 1 is therefore to perform breast biopsy procedures. In
The apparatus 1 further comprises a breast positioning assembly 4 conceptually consisting of an upper contact element 41 and a lower contact element 42 between which a breast M is compressed. In particular, the element 42 is fixed with respect to the detector 6 and the element 41 is movable with respect to the element 42 in that it can firstly move away to insert the breast M and then move closer by compressing the breast M. The element 41, called “compression plate” and hereinafter “compressor” for ease of brevity, may be adapted to receive electrical position control signals and/or transmit electrical measurement signals, such as force and/or position. In
In
No further details of the components mentioned above are provided herein because they are in themselves known in the field.
Two important components of the apparatus 1 shown in
Unit 7 is electrically connected to all the other components and is the one that basically determines the operation of apparatus 1. It is, in general, a computerised unit equipped with a system software and possibly an application.
Unit 9 is connected to unit 7 and allows the operator to provide “inputs”, e.g., data and/or commands, to the apparatus 1 in particular unit 7, and to receive “outputs” from the apparatus 1 in particular from unit 7. Typically, the unit 9 is a computerised unit equipped with an application software, in particular a PC, and comprises at least one screen, a keyboard and a mouse and/or joystick, as well as other devices for controlling, for example, the movements of the various components of the apparatus and the delivery of X-rays.
Typically, thanks to the appropriate software of the processing and control unit and/or thanks to the human-machine interface unit with relative appropriate software, it is possible to carry out the methods according to the present invention. The components of unit 7 and unit 9 are not shown in
While the breast M is compressed between elements 41 and 42:
Delivering X-rays on various occasions for acquiring the above 3D image is performed using a certain voltage (anodic), which is always the same, and with a certain filtering, which is always the same.
Rather than the point of the detector 6 in the acquisition plane 61, another point can be used that can be located for example on a central axis perpendicular to the detector 6 preferably between the acquisition plane 61 and the plane associated with the element 41.
Performing a 2D DEM or CESM imaging procedure with the apparatus of
Typically, the unit 7 processes the two received 2D images and generates a composite 2D image comprising a 2D matrix of pixels corresponding to a combination of the first image and the second image.
The method according to the present invention will be illustrated hereinafter with the aid of
The method according to the present invention is for aiding an operator to determine a spatial position of a region of interest in a breast (M) in order to subsequently guide a biopsy procedure in the breast itself, for example, to firstly move appropriately the biopsy assembly (8) and then to withdraw tissue by inserting and extracting the biopsy needle (81).
In general, the method comprises the steps of:
It should be noted that the 2D imaging procedure and the 3D imaging procedure are performed on a breast in a compressed state resulting from a single compression operation, and wherein a contrast medium is present. Such breast preparation may be considered, for example, as an “initial activity” and may fall within block 410 of
It can be understood from the above that a way has been found to determine a precise relationship (combination of the geometric parametrics of mapping and geometric transformation) between the 2D mammographic dual-energy image and the 3D tomosynthetic image; such a relation passes through the virtual 2D image. The virtual 2D image can be thought of as a “virtual” mammography that is not performed on a physical breast, but on a 3D reconstruction of the breast obtained by tomosynthesis.
Once this precise relationship has been determined, it is possible to switch precisely from dual energy mammography to tomosynthesis and vice versa; in particular, to display corresponding points and to know their spatial position, i.e., their coordinates.
This is useful when preparing for a biopsy to ensure that the biopsy needle withdraws a sample of tissue exactly at the point of the breast deemed as most relevant by the operator. Thus, the marking point referred to in step F of the method is conceptually related to the position of interest.
Suppose, for example, that a hypothetical lesion has been identified in the recorded dual-energy mammography (this could derive from an operator observing it) and there is interest in identifying the lesion in the tomosynthesis and its spatial position; the operator may create, for example, a marking point on the 2D mammography image by selecting the point with a mouse (which will be highlighted); then, the operator can view one or more of the sections of the 3D tomosynthetic image and, thanks to the method according to the present invention, the marking point is automatically displayed in such sections; finally, the operator can evaluate if such marking point is suitable and signal it to the apparatus as a point to be reached with the biopsy.
From this example, it can be understood that the operator may be interested in operating (i.e., identifying marking points such as selecting points and/or moving points that have already been selected with the mouse) in the mammographic image and/or tomosynthetic section images. According to preferred embodiments of the present invention, both things are possible.
The aid method according to the present invention may provide displaying images on one or more screens; for example, it may be provided that a mammographic image (with possible marking elements) and an image of a tomosynthetic section (with possible marking elements) are displayed simultaneously on one or two screens or alternately on the same screen.
Typically, according to the present invention, steps A-F are performed in the order indicated above; indeed, it is very advantageous to perform mammography before tomosynthesis. However, changes of order are not excluded; for example, step B could be performed after step C.
It should be considered that the core of the present invention corresponds to steps D-F. In other words, according to certain embodiments, the aid method according to the present invention may start from a 2D mammographic dual energy image already acquired and from a 3D tomosynthetic image already acquired, under the assumption that they refer to the same breast in the same position, that is, without macro-movements of the breast as it always remains compressed.
According to the present invention, as is common practice, the 2D imaging procedure and the 3D imaging procedure are performed on a breast in a compressed state. Advantageously, according to the present invention, the 2D imaging procedure and the 3D imaging procedure (at least those used for steps E and F of the method) are performed on a breast in a compressed state resulting from a single compression operation.
Advantageously, according to the present invention, the 2D imaging procedure and the 3D imaging procedure (at least those used for steps E and F of the method) are performed on a breast in which a contrast medium is present. In particular, a dose of a contrast medium may be injected only once before the 2D imaging procedure because the timing of steps A, B and C is such that the contrast medium remains in circulation and both can benefit (even if steps A and B are repeated two or three times).
Preferably, according to the present invention, the 2D imaging procedure is carried out with the use of an X-ray grid, which is then removed before carrying out the 3D imaging procedure.
The method according to the present invention advantageously provides one or more repetitions of a performance of a 2D imaging procedure according to step A and a generation of a 2D composite image according to step B (i.e., repetitions of one or more dual energy mammographies) prior to performing steps A, B, C, D, E, F. Each of these repetitions typically derives from a specific request of an operator and is typically preceded by a preliminary operation of repositioning the breast with a subsequent preliminary operation of compressing the breast (which is distinct from the only compression operation mentioned above and on the basis of which steps D and E and F of the method are carried out).
It is useful to carry out a repetition if it is estimated that the biopsy procedure will not be positively reached. For example, an operator might look at a mammographic image and identify a hypothetical lesion that should be biopsied; then the operator, possibly assisted by a computer program, might estimate whether it is reachable, taking into account for example the estimated spatial position of the breast, the biopsy assembly and the position(s) of the access/withdrawal window(s) of the compressor; if this estimate is negative, i.e. the lesion is not reachable, the breast must be repositioned and the dual energy mammography must be repeated. According to the present document, as is common practice, the second energy mentioned in step A is greater than the first energy mentioned in step A; conceptually, it is irrelevant in a dual-energy mammography to first perform a “low energy” acquisition and then a “high energy” acquisition or vice versa; in practice, it may be preferable to start with the “low energy” acquisition (which also implies a lower total energy in the detector) to avoid “ghost” effects particularly evident in some types of X-ray detectors.
Preferably, according to the present invention, the third energy mentioned in step C is equal to or about equal to the second energy mentioned in step A, i.e., “high energy”; for example, the third energy may be lower than the second energy by, for example, 10-30%. In the case of different energies, the tomosynthesis imaging is performed at an energy such that there is a peak absorption of photons by the contrast medium, neglecting the optimisation of the visibility of the surrounding tissues (in contrast to conventional tomosynthesis). In this case, it may be appropriate to determine the transformation in step E of the method using, for example, the “high energy” image instead of the composite image.
In explaining step E of the method, reference was made to portions of the images to determine the transformation; this allows for instance to exclude irrelevant parts of the images (such as the “background”) and/or to take in consideration only parts that are highly relevant for the biopsy. The portion of the image used in step E can be defined by the operator by, for example, drawing the outline of this portion on the screen. Alternatively, the portion of the image used in step E may be defined by a computer program and may correspond, for example, to an area of maximum reachability of the biopsy procedure, in particular enclosed by the profile of an access/withdrawal window of the compressor displayed on the screen.
In case of dual energy mammographies being repeated, it is preferable that the transformation referred to in step E of the method is determined on the basis of a 2D image deriving from the last performance of these two steps prior to step E, in particular the last composite 2D image generated or the last “high energy” 2D image acquired; in this way, there will be a greater relation between the 2D image and the 3D image.
Previously, when generally describing the method according to the present invention, reference was made to “at least one marking point”. However, it is typical and advantageous of the present invention that in step F the operator and/or the computer program identifies (and highlights) a marking area surrounded by a closed marking line. In this case, the display can be focused on, for example, and be limited to the closed marking line.
According to the present invention, it is possible and advantageous that geometric mapping parametrics are used to display in one or more sections of the recorded 3D image and/or in the 2D image points, lines or areas belonging to a volume calculated by a computer program; this may be useful, for example, to display the (simulated) passage cylinder of the biopsy needle.
Thus, in general according to the present invention, it is possible to display on a screen the recorded 2D image and/or one or more sections of the 3D image as well as a marking point or a marking line or a marking area.
We will now consider more in detail image processing.
In step D, mapping may include:
Such a function may be “binary” (i.e., defined over two values) or “numeric” (i.e., defined over a discrete range of values) or a mixture of “numeric” and “binary” elements—since processing according to the present invention will be carried out by a computer, strictly speaking, one cannot speak of a “continuous range of values”.
This function may be a combination (e.g., a linear combination) of maximum and/or minimum and/or mean and/or median.
In step E, the transformation may be of the geometric type; in particular, it may comprise: translation and/or rotation and/or uniform scaling and/or non-uniform scaling and/or shear deformation and/or perspective deformation (also called homographic transformation which is much preferred).
In step E, the transformation may be determined by a preliminary feature extraction from a portion of the reference 2D image (in particular the composite 2D image or the “high energy” 2D image) and from the corresponding portion of the virtual 2D image; consider for example the “high energy” image 1002 in
The algorithm that allows the transformation to be achieved is complex and typically includes: a step of extracting the relevant characteristics or “features” from the two images, a step of creating the couplings for the features extracted from the two images and a step of identifying the geometric transformation that adapts one image to the other according to the couplings created.
Two aspects of the algorithm need to be considered with particular care during practical implementation: the choice of a number of “features” to be extracted from the two images and the constraints to be imposed on the “feature” couplings in order to filter out false matches. As the number of “features” increases, if the couplings are sufficiently reliable according to the constraints imposed, the accuracy with which the transformation is identified increases, but so does the time required to calculate the couplings.
According to the present invention, a “feature” extraction method of the ORB (=Oriented Fast and Rotated Brief) type is preferably employed in which a maximum number of “features” equal to about 1% of the total pixels composing the image under examination is preferably set, which is preferably segmented with respect to the background of the image (typically, therefore the background pixels are not considered); in order to reduce the calculation time, this limit could also be reduced for example to 0.1%. As already mentioned, extracting the features from an image under examination may only concern a portion of the image.
The ORB method basically consists of a combination of two algorithms called FAST (=Features from Accelerated Segment Test), which deals with “feature” extraction and BRIEF (=Binary Robust Independent Elementary Feature), which deals with the description of each individual feature. The FAST algorithm is based on the detection of contrast differences for each pixel in the image to which a “feature” will eventually be associated, and also allows features of different shapes and sizes to be identified thanks to a multi-scale analysis of the image (FAST can be iterated several times, each time sub-sampling the original image according to a geometric pyramid model). The BRIEF algorithm assigns a string of bits to each identified feature based on, for example, the shape and size of the feature, which constitutes the feature descriptor; the advantage of this bit-string representation is that feature descriptors can be processed more efficiently during the coupling process.
In order to determine the matches between the identified features, a matching method may advantageously be used that compares the descriptors (in pairs and in such a way that there is always a descriptor for each of the two images) for all the features identified in the two images and creates a coupling based, as a quantitative evaluation criterion, on the distance measure between the descriptors; preferably, the Hamming distance between two bit strings (which encode the descriptors) is calculated.
These methods are generally applicable to the analysis of any images and, therefore, the resulting matches do not take into account specific aspects related to, for example, the typical size of the lesion of interest and/or the typical size of the anatomical structures involved and/or their positioning. Therefore, according to the present invention, which is in the specific field of breast imaging, it is advantageous to provide for an additional filtering step which eliminates all obtained couplings not satisfying a number of spatial constraints. Among them, preferably an upper limit is set to the (Euclidean) distance for pairs of corresponding features (i.e. coupled according to the previous calculations) within the area of the breast under consideration in order to restrict the analysis only to consistent displacements with respect to the real movement of breast tissues during compression; this constraint can also preferably be obtained by estimating a weighted value (e.g. average displacement) from the analysis of the distribution of dislocations for pairs of corresponding features identified in the area of the breast under consideration. Feature pairs that do not fulfil the imposed constraint criteria are then discarded from the analysis, which leads to obtaining the transformation, ensuring a greater reliability.
Once the corresponding feature pairs of interest have been defined, in order to obtain the transformation used for recording, two vectors, usually called “source” and “target”, are constructed, which contain the points with the two-dimensional coordinates of the features belonging respectively to the source image (e.g., the composite 2D image or a portion thereof—see image 1003 of
In the image above (
The method according to the present invention is specifically adapted to be implemented in a mammography apparatus.
For example, the apparatus 1 in
The core of the method for aiding an operator according to the present invention described above consists of steps D, E and F.
It can thus be defined as a computer-aided method which is carried out entirely by a computer electronic unit (such as a PC) and which comprises only steps D, E and F (and possibly also step B) and which operates on images stored in advance in a memory, e.g., an internal electronic or magnetic or optical memory or an external electronic or magnetic or optical memory. In this case, steps A and C (and possibly also step B) wherein there is (electromagnetic) interaction with a breast are carried out in advance (e.g., a few seconds before, or even less).
Number | Date | Country | Kind |
---|---|---|---|
102020000021274 | Sep 2020 | IT | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/IB2021/058115 | 9/7/2021 | WO |