Patent Application
20230147764
The present invention relates to a method of calculating a scaled virtual grid for x-ray projection images, a device for calculating such a scaled virtual grid for x-ray projection images, a system for calculating such a scaled virtual grid for x-ray projection images and a program element for such a method.
Mobile X-ray fluoroscopy for imaging of objects is known in the state of the art as well as widely used in various fields such as e.g. orthopedics, trauma, vascular and spine. Due to their design mobile x-ray fluoroscopies are characterized compared to fixed x-ray systems by a small footprint, a high maneuverability and reduced costs. Depending on the information required of the object sometimes several pictures have to be taken and analyzed. Physicians use these pictures as basis for their preparation of surgical procedures to get size information about proper implants, e.g. screws. Therefore normally an object of known size is overlapped over a desired anatomy of the patient during X-ray imaging. In case the object is not plane parallel to the detector or if area of the anatomy lays not in the plane of the object, the estimated distances are less reliable and the measurement lacks of accuracy.
For the inventors of the present invention, it has now become apparent that there is a need for improving the analysis of x-ray images of such mobile x-ray fluoroscopies. In particular there is a need to increase the efficiency of the geometrical information gained by the analysis of x-ray images of such a mobile x-ray fluoroscopy.
It is therefore the task of the present invention, to provide a method and a system, which enable an efficient analysis of x-ray images of, for example, such a mobile x-ray fluoroscopy. This task and other tasks, which are further mentioned when reading the following description or can be recognized by the skilled person, are solved by the subject-matter of the independent claims. Further embodiments and advantages of the invention are incorporated in the dependent claims.
The described embodiments similarly pertain to the method, the calculation device, the system and the computer program element and the computer-readable medium. Synergetic effects may arise from different combinations of the embodiments although they might not be described in detail.
Further on, it shall be noted that all embodiments of the present invention concerning a method, might be carried out with the order of the steps as described, nevertheless this has not to be the only and essential order of the steps of the method. The herein presented methods can be carried out with another order of the disclosed steps without departing from the respective method embodiment, unless explicitly mentioned to the contrary hereinafter.
Technical terms are used by their common sense. If a specific meaning is conveyed to certain terms, definitions of terms will be given in the following in the context of which the terms are used.
According to a first aspect of the present invention, a method for calculating a scaled virtual grid for x-ray projection images is provided. The method comprises providing at least a first and a second co-registered x-ray projection image Ij, j=1,2, . . . of a desired anatomy of a patient (step S1), defining two points P1 and P2 of the desired anatomy in 3-D space and determining 3-D coordinates of the two points P1 and P2 thereby using the x-ray projection images (step S2), calculating the scaled virtual grid based on the determined 3-D coordinates of the two points P1 and P2 (step S3), and projecting and displaying the calculated grid to a user on an x-ray projection image, preferably at least one of the first and the second co-registered x-ray projection image (step S4).
The term “scaled virtual grid” is to be understood broadly in the context of the present invention and comprises rectangular and distorted grids with a scale. The term “scale” is also to be understood as a metric of the virtual grid and allows a user/viewer to derive directly measurement values, such as distances, from the x-ray images of the desired anatomy imaged by the at least two projection images. Hence, there is no need for the viewer to estimate a distance in the x-ray image, which implies an increase of the analysis accuracy. Alternatively, the analysis of the scale can also carried out by an algorithm. Also a combination of determining measurement values by the user and by an algorithm are of course possible. It should be noted that determining or deriving measurement values particularly comprises determining or deriving length or distance measurement values. Further details and embodiments of the scaled virtual grid used in the present invention will be explained hereinafter in the context of embodiments and e.g. in the detailed descriptions of the embodiments depicted in
Co-registration of images, particularly of x-ray projection images, is state of the art and is widely used in image acquisition and evaluation. The provision of such co-registered images can comprise the use of co-registered x-ray projection images of external sources, like e.g. medical data bases, other imaging devices etc., as well as self-executed co-registration by means of an algorithm, and/or by the system carrying out the method present herein. Further descriptions of the provision of co-registered x-ray projection images will be explained hereinafter in the context of particular embodiments and e.g. the detailed description of the embodiment shown in
As is apparent to the skilled reader, image registration is the process of transforming different images/sets of data into one coordinate system. The images/data can be multiple photographs and/or data from different sensors, different times or different viewpoints. It is used in computer vision, medical imaging and in compiling and analyzing images and data from satellites. Registration is necessary in order to be able to compare or integrate the images/data obtained from these different measurements. Thus, co-registered x-ray projection images as used herein, are images which have been transformed into one coordinate system.
The desired anatomy object to be imaged can have any geometry. The definition of the points P1 and P2 and the determination of their 3-D-coordinates can be carried out by the user itself, who is interested in the analysis of the desired object's anatomy, or alternatively by or in combination with an algorithm. Such an algorithm may use patient data and/or an anatomy data base to define the two points P1 and P2 and the determination of their 3-D-coordinates. The user may e.g. provide a user input such that the algorithm automatically identifies the Femur Head in the first image, i.e. thus determining automatically P1 and its 3D coordinates, as well as P2 with respect to the second image.
Speaking generally, the knowledge of the 3-D coordinates enables an exact determination of lengths/distances in the x-ray projection images and not only distance between the two points P1 and P2. Further descriptions of possible ways of defining the two points P1 and P2 will be provided hereinafter in the context of particular, non-limiting embodiments and e.g. the detailed description of the embodiment of
As will become apparent from the detailed description hereinafter, in one embodiment three points P1, P2 and P3 are defined and their 3D coordinates are determined for calculating the grid.
Moreover, the calculation of the scaled virtual grid can be carried out by various mathematical methods, as will become apparent from the explanations hereinafter in the description of the particular embodiments. In examples, control points ci and cross lines dij can be calculated in various different manners for determining the scaled virtual grid.
Projecting and displaying the calculated grid to a user on at least one of the first and the second co-registered x-ray projection image can be carried out on one or more/various displays, e.g. using an HMI of a CT, and/or static screens and portable smart glasses using Augmented Reality technologies.
This aspect of the present invention is based on the insight of the inventors that the use of objects, which are placed on the desired anatomy during CT-imaging only provides exact measurement data for regions, which lay parallel in the plane of object of known size, wherein the object of known size also has to lay in a plane parallel to the x-ray detector. Hence, every measurement of a distance outside of the plane of the object of known size, wherein the object of known size lays in a plane parallel to the x-ray detector, leads to an unavoidable inaccuracy. These disadvantages is overcome by the present invention. For example, in an especially preferred embodiment of the invention it is possible to measure the distances in x-ray projection images without the use of an additional object of known size for any perspective of the anatomy. The distances in regions of interest to be measured can lay outside a plane parallel to the detector without losing measurement accuracy. Hence, with the use of the grid calculated according to the present invention, the physician can always derive reliable measurement values from the x-ray projection image without an explicit calibration step.
It should be noted that in the context of the present invention the following index convention is used: j represent the image index; i represents reference point index; m represents the control point index on cross line ij. This will become apparent from the following description.
According to an exemplary embodiment, the step of calculating the scaled virtual grid (step S3) comprises: calculating at least two control points ci; i=1,2, . . . on a line connecting the defined points P1 and P2 (step S3a); calculating for each co-registered image cross lines dij; i=1,2, . . . ; j=1,2, . . . (step S3b), wherein each cross line dij crosses the line connecting the defined points P1 and P2 at control point ci, wherein each cross line dij is parallel with a detector, with which image Ij was captured, and preferably each cross line dij is perpendicular to the line connecting the defined two points P1 and P2.
In a preferred embodiment the step of calculating on each cross line dij control points kijm (S3c), wherein the control points kijm define the scaled virtual grid for each image Ij.
With each cross lines dij being parallel with the detector that was generating the corresponding image Ij, the scale of the grid is equidistant and does not change the magnification. Further detailed embodiments that build on this way of calculating the grid can be gathered e.g. from the particular embodiment explained in the context of
Alternatively, the grid can also be defined with three selected points P1, P2 and P3, which define a plane, in which the cross lines dij lie. The cross lines dij are perpendicular to the line connecting the defined points P1 and P2, but not parallel to the detector. This can be an advantage if your region of interest is located in special plane of the object.
In other words, another way to define the grid is to define/select a third anatomical point P3 in the two x-ray images. P3 together with P1 and P2 define a plane. Then we one use steps S1-S3a, as has been explained in detail hereinbefore and will be elucidated even more hereinafter, to define ci. Then one can uniquely build cross lines dij in the plane defined by the three points P1, P2 and P3. Such a dij crossline is perpendicular on P1P2 line, but it is not parallel anymore with the detectors, as was the case in the embodiment described before. In this approach, the grid is calibrated and attached to the plane defined by P1P2P3, as is clear to person skilled in the art from this disclosure.
According to another exemplary embodiment of the invention the method comprises the steps of projecting and displaying the scaled virtual grid, which is defined by control points ki1m on the first co-registered x-ray projection image I1 to the user, and projecting and displaying the scaled virtual grid, which is defined by control points ki2m on the second co-registered x-ray projection image I2 to the user.
In other words, in this embodiment the calculated grid is laid over the first and second co-registered x-ray projection images I1 and I2 to be used by the user on a display. Hence, the user/viewer has the scale information for the analysis of the region of interest in both co-registered x-ray projection images I1 and I2.
According to another exemplary embodiment of the invention, the method comprises calculating the distance in 3-D space between the two points P1 and P2, and using the calculated distance to determine the distance between at least two control points kijm.
The scale of the grid is calculated in cross direction and in longitudinal direction with the determination of the distances between all control points kijm. The distances between two neighboring control points kijm can have any values, but preferably they have equidistant values. It may be the case that humans can recognize or analyze easier grids with an equidistant grid size.
According to another exemplary embodiment of the invention, the method comprises displaying the determined distance between the at least two control points kijm together with the control points kijm on the image Ij to the user.
The distances between the control points kijm may be understood to represent the grid and serve the viewer as scale and/or metric. Further descriptions of displaying the determined distance between the control points kijm together with the control points kijm on the image Ij to the user can easily gathered from the embodiment described and shown with
According to another exemplary embodiment of the invention, the first and second co-registered x-ray projection images show the desired anatomy of the patient from different perspectives.
The different perspectives enable various information of the region of interest, e.g. information, which is hidden in one perspective, is visible in another perspective. The different perspectives are the basis for the necessary alignment of the points P1 or P2 in the co-registered x-ray projection images, which serves further the determination of the 3-D coordinates of the points P1 and P2 in space. Further descriptions of the different perspectives can be found in embodiments hereafter and in the detailed description of
According to another exemplary embodiment of the invention, the method comprises localizing an x-ray imaging device in six degrees of freedom, wherein the provided first and second co-registered x-ray projection images were generated with said x-ray imaging device.
The localizing or localization of the x-ray imaging device is a requirement for the co-registration of the x-ray projection images, which are further processed for generating the virtual grid. X-ray imaging devices are well known in the state of the art, and examples will be mentioned hereinafter briefly. In this embodiment the x-ray imaging device comprises a source and corresponding a detector, which can move in dependence to each other in 6 degrees of freedom. This means that there is a fixed geometric constraint between the source and the detector. The anatomy object is placed between the x-ray source and the x-ray detector. Hence, it is possible to align the images of the x-ray imaging device with the information of the position of the x-ray imaging device comprising 6 degrees of freedom.
According to another exemplary embodiment of the invention, the localization is carried out using a ripple marker, using optical tracking of the x-ray imaging device, using internal encoders of the x-ray imaging device, and/or using other image-based markers.
Hence, there are different opportunities for the same aim, i.e. the localization of the x-ray imaging devices in space, which can be used single or in combination for the sake of protection through redundancies.
In another exemplary embodiment of the method the step of defining two points P1 and P2 of the desired anatomy in 3-D space (step S2) comprises identifying a first landmark of the desired anatomy in the first x-ray projection image (step S2a), determining a first line on which the first landmark lies between an x-ray source and an x-ray detector that were used for generating the first x-ray projection image (step S2b), identifying a second landmark of the desired anatomy in the second x-ray projection image (step S2c), determining a second line on which the second landmark lies between the x-ray source and the x-ray detector that were used for generating the second x-ray projection image (step S2d), and calculating 3-D coordinates of an intersection of the determined first and second lines (step S2e), wherein the calculated 3-D coordinates define point P1 in 3-D space; identifying a third landmark of the desired anatomy in the first x-ray projection image (step S2f), determining a third line on which the third landmark lies between the x-ray source and the x-ray detector that were used for generating the first x-ray projection image (step S2g), identifying a fourth landmark of the desired anatomy in the second x-ray projection image (step S2h), determining a fourth line on which the fourth landmark lies between the x-ray source and the x-ray detector that were used for generating the second x-ray projection image (step S2i), and calculating 3-D coordinates of an intersection of the determined third and fourth lines (step S2j), wherein the calculated 3-D coordinates define point P2 in 3-D space.
It should be noted that said steps of this embodiment, particularly the steps of identifying said landmarks and determining said lines can be carried out manually, e.g. based on a user input via e.g. a HMI, but can also be carried out purely automatically using e.g. image processing algorithms. Of course, also a combination of both alternative ways can be used without departing from the preset invention. Moreover, a particularly detailed embodiment of how the two points P1 and P2 of the desired anatomy in 3-D space can be defined using the identification of said landmarks and the determination of said lines can be gathered from e.g.
In another exemplary embodiment, the method comprises projecting the determined first line onto the second x-ray projection image, and/or projecting the determined third line onto the second x-ray projection image.
It shall be mentioned, that the step of defining the two points P1 and P2 is not limited to lines as geometric tools, as is clear to the skilled reader. Also other geometric forms e.g. circles, triangles etc. are possible to carry out the method. Further explanation of defining the points can be found in the detailed description of
In another exemplary embodiment the projection of the determined first and/or third line is used as a constraint, e.g. on a Human Machine Interface (HMI), which constraint constrains user input possibilities for identifying a landmark in the second x-ray projection image, e.g. on said HMI.
The constraint limit may limit the solution space for an alignment of e.g. P1 in the first and second x-ray projection image, while defining or identifying P2. Hence, the constraint assists the user of HMI and reduces the possible error rate or increases the accuracy of the scale of the grid. As mentioned above, it is also possible to automate the definition of the points P1 and P2 by an algorithm based on patient data or anatomical database, which does not require a user input via a HMI.
According to another exemplary embodiment, the method comprises the step of generating the first and second x-ray projection images with an x-ray imaging device, preferably with a mobile C-arm CT, comprising an x-ray source and an x-ray detector.
Mobile C-arm CT are well known from the state of the art and allow the user to easily generate images of different perspectives of the desired object. Further explanations about generating the first and second x-ray projection image can be found in the detailed description of the embodiment shown in
According to further aspect of the present invention a device for calculating a scaled virtual grid for x-ray projection images is presented. The device comprises a calculation unit configured for: receiving at least a first and a second co-registered x-ray projection image Ij of a desired anatomy of a patient, calculating two points P1 and P2 in the x-ray projection images and in the desired anatomy thereby determining 3-D coordinates of the two points P1 and P2, calculating the scaled virtual grid based on the determined 3-D coordinates of the two points P1 and P2, and causing a projection and a displaying of the calculated grid to a user on at least one of the first and the second co-registered x-ray projection image.
Such a device may be e.g. a processor that can be part of an x-ray imaging device, such as a mobile C-arm CT, comprising also an x-ray source and an x-ray detector. But said device for calculating a scaled virtual grid for x-ray projection images may also be a separate component that receives as an input the images and delivers as an output the calculated grid. Moreover, the calculation device may itself project and display the calculated grid to the user on at least one, preferably both, co-registered x-ray projection images. Alternatively, the calculation device may generate e.g. one or more control signals that cause another device, like e.g. a projector or a display to carry out the projection and the displaying of the calculated grid to the user on one or both or more co-registered x-ray projection images.
A further aspect of the present invention relates to a system for calculating a scaled virtual grid for x-ray projection images comprising a device for a scaled virtual grid for x-ray projection images as mentioned hereinbefore. It also comprises a Human Machine Interface (HMI) configured for displaying the received first and a second co-registered x-ray projection images and configured for receiving input signals from the user e.g. for identifying landmarks in the displayed images of the desired anatomy.
According to another exemplary embodiment, the system comprises an x-ray imaging device, preferably a mobile C-arm CT, comprising an x-ray source and an x-ray detector for generating the x-ray projection images.
It should be noted that the afore mentioned parts of the system can be arranged centrally or decentralized to each other. Centrally means in that all parts of the system are at least in a wired connection and decentralized means that the parts can be elsewhere and communicate with each other via a wireless network.
A further aspect of the present invention relates to a program element, which, when run on a processor or computer, is configured to carry out the method for calculating a scaled virtual grid for x-ray projection images as described herein.
The program element may be part of a computer program, but it can also be an entire program by itself. For example, the program element may be used to update an already existing computer program to get to the present invention.
A further aspect of the present invention relates to a computer readable medium, on which such a program element, as described before, is stored.
The computer readable medium may be seen as a storage medium, such as for example, a USB stick, a CD, a DVD, a data storage device, a hard disk, or any other medium on which a program element as described above can be stored.
In the following, the disclosure is described exemplarily with reference to the enclosed figures, in which
Notably, the figures are merely schematic representations and serve only to illustrate embodiments of the present disclosure. Identical or equivalent elements are in principle provided with the same reference signs.
In other words, in step S1 at least two co-registered x-ray projections of a desired anatomy from different perspectives are be provided and further used for the calculation the scaled grid. The co-registered x-ray projections serve as data basis for the following steps. The different perspectives of the co-registered x-ray projections are necessary, for the second step S2, wherein two points are defined respectively selected in the region of interest in the co-registered x-ray projection image. For determining the 3-D coordinates of these two points an alignment of each selected point with the corresponding position in the other co-registered x-ray projection image is carried out. With this position information of each point in both co-registered x-ray projection images it is possible to determine the 3-D coordinates of each point in the 3-D space of the desired anatomy. These 3-D coordinates of the two points serve as base for a scale grid of the anatomy, which is designed in step S3. In the last step S3 the designed scale grid is projected on co-registered x-ray projections, which means it is overlaid on the co-registered x-ray projections. Hence, there is a calibrated scale grid overlaid on both co-registered x-ray projections, which is displayed to a user.
Note that the calculation of the scaled virtual grid can be carried out by various mathematical methods, as will be elucidated with two embodiments explained in the context of
This method is based on the insight of the inventors that the use of objects, which are placed on the desired anatomy during CT-imaging only provides exact measurement data for regions, which lay parallel in the plane of object of known size, wherein the object of known size also has to lay in a plane parallel to the x-ray detector. Hence, every measurement of a distance outside of the plane of the object of known size, wherein the object of known size lays in a plane parallel to the x-ray detector, leads to an unavoidable inaccuracy. These disadvantages is overcome by the method of
In other words, a line is drawn between the two points P1 and P2, wherein the line serves a center line in the grid. The line is further divided in part sections, defined by the point ci (step S3a). These part sections define the grid size in one direction. The calculated crosslines dij run through the points ci and are perpendicular to the center line and parallel to each other (step S3b). Each cross line dij shows calculated points kijm(step S3c). These points define the mesh size in the second direction. As a result, you gain the calibrated scale grid for x-ray projection images for the desired anatomy.
Alternatively, the grid can also be defined with three selected points P1, P2 and P3, which define a plane, in which the cross lines dij lie. The cross lines di are perpendicular to the line connecting the defined points P1 and P2, but not parallel to the detector. This can be an advantage if your region of interest is located in special plane of the object. In other words, another way to define the grid is to define/select a third anatomical point P3 in the two x-ray images. P3 together with P1 and P2 define a plane. Then we one use steps S1-S3a, as has been explained in detail hereinbefore and will be elucidated even more hereinafter, to define ci. Then one can uniquely build cross lines dij in the plane defined by the three points P1, P2 and P3. Such a dij crossline is perpendicular on P1P2, but it is not parallel anymore with the detectors, as was the case in the embodiment described before. In this approach, the grid is calibrated and attached to the plane defined by P1P2P3, as is clear to person skilled in the art from this disclosure.
The system further comprises a calculation unit 15, which is configured for receiving at least a first and a second co-registered x-ray projection image Ij of a desired anatomy of a patient, calculating two points P1 and P2 in the x-ray projection images and in the desired anatomy thereby determining 3-D coordinates of the two points P1 and P2. The calculation unit can calculate the scaled virtual grid based on the determined 3-D coordinates of the two points P1 and P2, and causes projection and a displaying of the calculated grid to a user on at least one of the first and the second co-registered x-ray projection image. In other words the calculation unit 15 is responsible for all steps of the calculation of the grid and the control of the projection and displaying the grid to a user.
In other words the desired anatomy 23 lies between x-ray sources 21, 22 and corresponding x-ray detectors 24, 25. The lines 33, 34, 36 and 37 shall give information about the optical path between the x-ray sources 21, 22 and the corresponding x-ray detectors 24, 25. The two x-ray sources 21, 22 and corresponding x-ray detectors 24, 25 are in reality one x-ray source and one x-ray detector of the same C-arm CT, which merely differ in their perspectives of desired anatomy. On the position of the x-ray detector 24 the x-ray projection image 26 from a first perspective is shown, which shall clarify that the x-ray projection image always correlates with a corresponding x-ray source/x-ray detector position, which is gained via a ripple marker 32. In reality, the user selects via HMI (not shown) on which the x-ray projection image 26 is displayed a point/landmark 29. In a next step the calculation unit (not shown) calculates a first line 35 between the point/landmark 29 and the x-ray source 22. In a next step a part of the first line 35, which lies between the lines 33 and 34, is projected on the second x-ray projection image 27. The second x-ray projection image corresponds to the x-ray source 21 and the x-ray detector 25 in the second perspective of the desired anatomy 23. The projected line 30 on the second x-ray projection image 27 serves as constraint, on which the landmark/point 28 can lie, for the user's selection of the landmark/point 28. The two landmarks/points 28, 29 represent the same feature of the desired anatomy only from different perspectives. After the selection of the landmark/point 28 in the second x-ray projection image 27, a line 31 is calculated between the x-ray source 21 and the landmark/point 28. The 3-D coordinates of the landmark/point in 3-D space are derived from the intersection of the lines 31 and 35.
Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from the study of the drawings, the disclosure, and the appended claims. In the claims the word “comprising” does not exclude other elements or steps and the indefinite article “a” or “an” does not exclude a plurality. A single processor or other unit may fulfill the functions of several items or steps recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored/distributed on a suitable medium such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be construed as limiting the scope of the claims.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2021/056537 | 3/15/2021 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 62992189 | Mar 2020 | US |
