Apparatus and method for image guided accuracy verification

Information

  • Patent Grant
  • 11304630
  • Patent Number
    11,304,630
  • Date Filed
    Friday, September 27, 2019
    5 years ago
  • Date Issued
    Tuesday, April 19, 2022
    2 years ago
Abstract
A method includes receiving during a first time interval associated with a path of motion of a dynamic body, image data associated with a plurality of images of the dynamic body. The plurality of images include an indication of a position of a first marker coupled to a garment at a first location, and a position of a second marker coupled to the garment at a second location. The garment is coupled to the dynamic body. During a second time interval, an image from the plurality of images is automatically identified that includes a position of the first marker that is substantially the same as a position of a first localization element relative to the dynamic body and a position of the second marker that is substantially the same as a position of the second localization element relative to the dynamic body.
Description
BACKGROUND

The invention relates generally to a medical device and particularly to an apparatus and method associated with image guided medical procedures.


Image guided surgery (IGS), also known as image guided intervention (IGI), enhances a physician's ability to locate instruments within anatomy during a medical procedure. IGS can include 2-dimensional (2-D) and 3-dimensional (3-D) applications.


Existing imaging modalities can capture the movement of dynamic anatomy. Such modalities include electrocardiogram (ECG)-gated or respiratory-gated magnetic resonance imaging (MRI) devices, ECG-gated or respiratory-gated computer tomography (CT) devices, and cinematography (CINE) fluoroscopy. The dynamic imaging modalities can capture the movement of anatomy over a periodic cycle of that movement by sampling the anatomy at several instants during its characteristic movement and then creating a set of image frames or volumes. Such images can be used to help a physician navigate a medical instrument to the desired location on the anatomical body during a medical procedure performed on the anatomical body at a later time.


Typical image-guided medical systems require manual user input to identify a pre-procedural image that corresponds to the same position and orientation of an anatomical body during a medical procedure. These manual operations can lead to greater errors and reduced efficiency in image-guided procedures.


Thus, a need exists for a method and apparatus that can automatically identify pre-procedural images of a targeted anatomical body that can be used to help a physician navigate a medical instrument to a selected location on the anatomical body during a range of motion of the anatomical body.


SUMMARY OF THE INVENTION

Apparatuses and methods for performing gated instrument navigation on dynamic anatomy with automatic image registration are disclosed herein. In one embodiment, a method includes receiving during a first time interval image data associated with a plurality of images of a dynamic body. The plurality of images includes an indication of a position of a first marker on a garment coupled to the dynamic body and a position of a second marker on the garment coupled to the dynamic body. The first marker is coupled to the garment at a first location, and the second marker is coupled to the garment at a second location. The first time interval is associated with a path of motion of the dynamic body. During a second time interval after the first time interval, data is received that is associated with a position of a first localization element relative to the dynamic body, and data is received that is associated with a position of a second localization element relative to the dynamic body. The first localization element is coupled to the garment at the first location, and the second localization element is coupled to the garment at the second location. The second time interval is associated with a path of motion of the dynamic body. During the second time interval, an image from the plurality of images is automatically identified that includes a position of the first marker that is substantially the same as the position of the first localization element relative to the dynamic body and a position of the second marker that is substantially the same as the position of the second localization element relative to the dynamic body.





BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is described with reference to the accompanying drawings.



FIG. 1 is a schematic illustration of various devices used with a method according to an embodiment of the invention.



FIG. 2 is a schematic illustration of various devices used with a method according to an embodiment of the invention.



FIG. 3 is a front perspective view of an apparatus according to an embodiment of the invention.



FIG. 4 is a graphical representation illustrating the function of an apparatus according to an embodiment of the invention.



FIG. 5 is a schematic illustration of an example of voxels of a connected-component in a 3-D volume according to an embodiment of the invention.



FIG. 6 is a schematic illustration of an example of voxels of a connected-component in a 3-D volume according to an alternative embodiment of the invention.



FIG. 7 is a flowchart illustrating a method according to an embodiment of the invention.



FIG. 8 is a schematic illustration of the flow of information during an automatic segmentation process.





DETAILED DESCRIPTION

A method according to an embodiment of the invention includes capturing images of a dynamic body during a path of motion of the dynamic body pre-procedurally (also referred to herein as “first time interval”). The images can be used to assist a physician in navigating a medical instrument to a desired location on the dynamic body during a medical procedure performed at a later time (also referred to herein as “second time interval”). The method uses a system configured to automatically perform segmentation, correlation and registration between data obtained in “model space” or “image space” (position data taken pre-procedurally) and data obtained in “physical space” (position data obtained during a later medical procedure).


Specifically, an apparatus is configured to be coupled to a selected dynamic body, such as selected dynamic anatomy of a patient. Dynamic anatomy can be, for example, any portion of the body associated with anatomy that moves during its normal function (e.g., the heart, lungs, kidneys, liver and vasculature). The apparatus can include, for example, two or more markers configured to be coupled to a patient and two or more localization elements configured to be coupled to the patient proximate the markers. In other embodiments, the apparatus can include, for example, a garment configured to be coupled to a patient, two or more markers coupled to the garment, and two or more localization elements coupled to the garment at a location proximate the markers.


A processor, such as a computer, is configured to receive the pre-procedural image data associated with the dynamic body taken during a pre-surgical or pre-procedural first time interval. The image data can include an indication of a position of each of the markers for multiple instants in time during the first time interval. The processor can also receive position data associated with the localization elements during a second time interval in which a surgical procedure or other medical procedure is being performed. The processor can use the position data received from the localization elements and the position data received from the images to automatically identify an image from the pre-procedural images where the position of the markers at a given instant in time during the pre-procedural imaging is substantially the same as the position of the localization elements corresponding to those markers, at a given instant of time during the later medical procedure.


A physician or other healthcare professional can use the images that were automatically identified by the processor during a medical procedure performed during the second time interval, such as, for example, an image-guided medical procedure involving temporal registration and gated navigation. For example, when a medical procedure is performed on a targeted anatomy of a patient, such as a heart, the physician may not be able to utilize an imaging device during the medical procedure to guide him to the targeted area within the patient. Markers or fiducials can be positioned or coupled to the patient proximate the targeted anatomy prior to the medical procedure, and pre-procedural images can be taken of the targeted area during a first time interval. The markers or fiducials can be viewed within the image data, which can include an indication of the position of the markers during a given path of motion of the targeted anatomy (e.g., the heart) during the first time interval. Such motion can be due, for example, to inspiration (i.e., inhaling) and expiration (i.e., exhaling) of the patient, or due to the heart beating. During a medical procedure, performed during a second time interval, such as a procedure on a heart, with the markers coupled to the patient at the same location/position as during the first time interval, the processor receives data from the localization elements associated with a position of the localization elements at a given instant in time during the medical procedure (or second time interval).


Because the markers are positioned at the same location on the patient relative to the dynamic body during both the first time interval and the second time interval, and the localization elements are coupled to the patient proximate the location of the markers, a correlation can be made between the position data in image space and the position data in physical space. For example, a position of the markers at an instant in time during the pre-procedural imaging corresponds to a specific position and orientation of the dynamic body at an instant in time during the path of motion of the dynamic body as viewed in the image data. When the medical procedure is performed during the second time interval, a position of the localization elements likewise corresponds to a specific positioning of the dynamic body at an instant in time during the path of motion of the dynamic body. Although the marker-localization element combinations can move relative to each other, for example, as the dynamic anatomy moves, the markers are in a fixed position relative to the patient during both the first time interval and the second time interval. As stated above, the localization elements are coupled to the patient proximate the markers, thus, when the position of the localization elements (identified during the medical procedure) is substantially the same as the position of the markers (identified in the image space), the image corresponding to that position of the markers is representative of the position of the dynamic body for that instant during the medical procedure.


An automatic segmentation-correlation-registration process can be performed after the image dataset is imported into the processor and the localization elements are connected to the processor. Once performed, the correlation does not change during the course of the procedure and the model space marker positions provide a baseline position for the temporal registration. After the segmentation-correlation and baseline registration has been computed, the localization element locations are sampled automatically and continuously to determine when the dynamic body is at or near the position at which the images were acquired. The affine rigid-body transformation is computed automatically and continuously until a temporal gating threshold is exceeded, indicating that the dynamic body is no longer near the same configuration as where the images were acquired. The automatic process produces simulated real-time, intra-procedural images illustrating the orientation and shape of the targeted anatomy as a catheter or similar structure is navigated to the targeted anatomy. Thus, during the medical procedure, the physician can view selected image(s) of the targeted anatomy that correspond to and simulate real-time movement of the anatomy.


In addition, during a medical procedure being performed during the second time interval, such as navigating an instrument, such as a catheter or needle to a targeted anatomy, the location(s) of an electromagnetic coil coupled to the instrumentation during the second time interval can be superimposed on an image of the instrumentation. The superimposed image(s) of the instrument can then be superimposed on the selected image(s) from the first time interval, providing simulated real time imaging of the instrument location relative to the targeted anatomy. This process and other related methods are described in pending U.S. patent application Ser. No. 10/273,598, entitled Methods, Apparatuses, and Systems Useful in Conducting Image Guided Interventions, filed Nov. 8, 2003, the entire disclosure of which is incorporated herein by reference.


Having described above various general principles, several example embodiments of these concepts are now described. These embodiments are only examples, and many other embodiments are contemplated by the principles of the invention, and will be apparent to the artisan in view of the general principles described above and the exemplary embodiments. For example, other possible embodiments can be used to perform some or all of the functions described herein, such as those systems and methods described in U.S. patent application Ser. No. 11/224,028, filed Sep. 13, 2005, entitled “Apparatus and Method for Image Guided Accuracy Verification” (referred to herein as “the '028 application”), the disclosure of which is hereby incorporated by reference in its entirety.



FIGS. 1 and 2 are schematic illustrations of devices that can be used to perform various procedures described herein. An apparatus 10 includes two or more markers or fiducials 22 coupled to a dynamic body B at selected locations, as shown in FIG. 1. The dynamic body B can be, for example, a selected dynamic portion of the anatomy of a patient. The markers 22 are constructed of a material that can be viewed on an image, such as an X-ray. The markers 22 can be, for example, radiopaque, and can be coupled to the dynamic body B using known methods of coupling such devices to a patient, such as with adhesive, straps, etc. FIGS. 1 and 2 illustrate the apparatus 10 having four markers 22, but any number of two or more markers can be used.


An imaging device 40 can be used to take images of the dynamic body B while the markers 22 are coupled to the dynamic body B, pre-procedurally during a first time interval. As stated above, the markers 22 are visible on the images and can provide an indication of a position of each of the markers 22 during the first time interval. The position of the markers 22 at given instants in time through a path of motion of the dynamic body B can be illustrated with the images. The imaging device 40 can be, for example, a computed tomography (CT) device (e.g., respiratory-gated CT device, ECG-gated CT device), a magnetic resonance imaging (MRI) device (e.g., respiratory-gated MRI device, ECG-gated MRI device), an X-ray device, or any other suitable medical imaging device. In one embodiment, the imaging device 40 is a computed tomography—positron emission tomography device that produces a fused computed tomography—positron emission tomography image dataset. The imaging device 40 can be in communication with a processor 30 and send, transfer, copy and/or provide image data taken during the first time interval associated with the dynamic body B to the processor 30.


The processor 30 includes a processor-readable medium storing code representing instructions to cause the processor 30 to perform a process. The processor 30 can be, for example, a commercially available personal computer, or a less complex computing or processing device that is dedicated to performing one or more specific tasks. For example, the processor 30 can be a terminal dedicated to providing an interactive graphical user interface (GUI). The processor 30, according to one or more embodiments of the invention, can be a commercially available microprocessor. Alternatively, the processor 30 can be an application-specific integrated circuit (ASIC) or a combination of ASICs, which are designed to achieve one or more specific functions, or enable one or more specific devices or applications. In yet another embodiment, the processor 30 can be an analog or digital circuit, or a combination of multiple circuits.


The processor 30 can include a memory component 32. The memory component 32 can include one or more types of memory. For example, the memory component 32 can include a read only memory (ROM) component and a random access memory (RAM) component. The memory component can also include other types of memory that are suitable for storing data in a form retrievable by the processor 30. For example, electronically programmable read only memory (EPROM), erasable electronically programmable read only memory (EEPROM), flash memory, as well as other suitable forms of memory can be included within the memory component. The processor 30 can also include a variety of other components, such as for example, co-processors, graphic processors, etc., depending upon the desired functionality of the code.


The processor 30 can store data in the memory component 32 or retrieve data previously stored in the memory component 32. The components of the processor 30 can communicate with devices external to the processor 30 by way of an input/output (I/O) component (not shown). According to one or more embodiments of the invention, the I/O component can include a variety of suitable communication interfaces. For example, the I/O component can include, for example, wired connections, such as standard serial ports, parallel ports, universal serial bus (USB) ports, S-video ports, local area network (LAN) ports, small computer system interface (SCSI) ports, and so forth. Additionally, the I/O component can include, for example, wireless connections, such as infrared ports, optical ports, Bluetooth® wireless ports, wireless LAN ports, or the like.


The processor 30 can be connected to a network, which may be any form of interconnecting network including an intranet, such as a local or wide area network, or an extranet, such as the World Wide Web or the Internet. The network can be physically implemented on a wireless or wired network, on leased or dedicated lines, including a virtual private network (VPN).


As stated above, the processor 30 can receive image data (also referred to herein as “image dataset”) from the imaging device 40. The processor 30 can identify the position of selected markers 22 within the image data or voxel space using various segmentation techniques, such as Hounsfield unit thresholding, convolution, connected component, or other combinatory image processing and segmentation techniques. The processor 30 can determine a distance and direction between the position of any two markers 22 during multiple instants in time during the first time interval, and store the image data, as well as the position and distance data, within the memory component 32. Multiple images can be produced providing a visual image at multiple instants in time through the path of motion of the dynamic body. The processor 30 can also include a receiving device or localization device 34, which is described in more detail below.


As shown in FIG. 2, during a second time interval, two or more localization elements 24 are coupled to the markers 22 for use during a medical procedure to be performed during the second time interval. The localization elements 24 are coupled to the patient adjacent the markers 22. The localization elements 24 can be, for example, electromagnetic coils, infrared light emitting diodes, and/or optical passive reflective markers. The markers 22 can include plastic or non-ferrous fixtures or dovetails or other suitable connectors used to couple the localization elements 24 to the markers 22. A medical procedure can then be performed with the markers 22 coupled to the dynamic body B at the same location as during the first time interval when the pre-procedural images were taken. During the medical procedure, the localization elements 24 are in communication or coupled to the localization device 34 included within processor 30. The localization device 34 can be, for example, an analog to digital converter that measures voltages induced onto localization coils in the field; creates a digital voltage reading; and maps that voltage reading to a metric positional measurement based on a characterized volume of voltages to millimeters from a fixed field emitter. Position data associated with the localization elements 24 can be transmitted or sent to the localization device 34 continuously during the medical procedure during the second time interval. Thus, the position of the localization elements 24 can be captured at given instants in time during the second time interval.


The image dataset, the position data for the markers from the first time interval (“model space”) and the position data for the localization elements during the second time interval (“physical space”) can be used to perform an automatic segmentation, correlation and registration between the data in the model space and the data in the physical space. The result of the analysis is to provide a physician with images that represent the position of a dynamic body during the second time interval when the physician is performing a medical procedure on the dynamic body. The processor 30 can be configured to perform the automatic segmentation-correlation-registration process as described in more detail below.


To identify actual position data associated with the markers 22 within the image dataset, the processor 30 can perform an automated segmentation procedure. Segmentation is the process of identifying reference points in the 3-D image dataset. The purpose of the segmentation is to automatically locate potential “landmarks” in the dataset that indicate a location where a marker 22 may be located. Segmentation can be performed in a variety of different manners. For example, a segmentation process can include, intensity filtering, connectivity analysis, and size and shape filtering to identify candidate sensor (e.g., marker) locations, or model space (also referred to herein as “image space”) coordinates of the marker 20 candidates. In some example embodiments, the intensity filtering applies domain knowledge to threshold the 3-D image dataset to select only those image values that fall within a designated intensity range that contains the reference points. For example, reference markers can be designated to appear in CT scans with Hounsfield units higher than the anatomical structures within the 3-D image. An example output from an intensity filtering process can include a 3-D binary volume with non-zero entries indicating voxels (i.e., a 3-D data point) with an intensity that falls within the range of values that characterize an image marker, as illustrated in FIG. 8. FIG. 8 is a schematic illustration of the flow of information during one example of an automatic segmentation process.


After filtering the image values based on intensity, a connectivity analysis can be performed. A connectivity analysis can use the output from the intensity filtering to analyze the potential candidates identified in the 3-D image dataset to identify, for example, “connected-components.” A connected-component, as used here, is a continuous 3-D region in the 3-D volume (i.e., image dataset) that is connected via adjacent voxels that are in contact with one another. Examples of voxels of connected-components are illustrated in FIGS. 5 and 6. FIG. 5 illustrates a connected-component having 8 connected voxel elements (indicated by the shaded boxes), and FIG. 6 illustrates a connected component having 4 connected voxel elements (indicated by the shaded boxes). From the identified connected-components, information about the connected regions, such as the location of each voxel element, the geometric center, and the volume and bounding perimeter dimensions can be identified. An example output of a connectivity analysis can include, for example, a list of each separate connected-component in the 3-D binary volume, and can be used in the next step in the segmentation process.


Next, in some embodiments, the output from the connectivity analysis, which includes the identified connected-components, can be filtered based on size and shape criteria during a size threshold analysis. First, knowledge about the size of the reference markers can be used to identify and discard any connected-components that are too small or too large to be valid markers. A list of connected-components that fulfill the size criteria can then be evaluated based on the shape of the components during a shape-filtering analysis. Knowledge about the shape of the reference markers can be used to discard any components that do not match the known shape of the reference markers. For example, if the markers are known to be cylindrical, then the connected component shape can be analyzed to determine if the ratio between the major axis and the minor axis is within a set criteria. The output from this step in this example process includes, for example, a list of connected-components that fulfill the shape criteria. Other analysis can be performed depending on the particular marker configuration, such as, for example, checking whether the connected-component shape is symmetric about a centroid of the connected-component.


After the segmentation process is performed, an automatic correlation process can be performed. Correlation as used here is the process of correctly matching reference points between the image or model space and the physical space. Correctly matching the reference points aids in accurately computing the registration between the data in the image space and the data in the physical space without user interaction. The correlation process determines where each of the localization elements is positioned in the model images. Correct correlation is required to compute an affine transform between model space and physical space. The apparatuses and methods described herein enable the process to be automated with minimal user intervention. Automatic correlation results in an understanding of the location of the markers in image space and physical space, as well as the corresponding labeling/identification of each marker in each space.


Because there are a large number of possible solutions, computations of all possible combinations can result in long computation times. According to an embodiment of the invention, the processor 30 can be configured to compute the correlation between the image space and the physical space at a much faster rate (e.g., 2 seconds on a 1.5 GHz G4 Macintosh computer).


Because the number of localization element positions in the physical space is typically smaller than the number of identified marker positions in the model space, a guess at a correlation can be made for three localization element points in physical space. An affine transform registration is then computed between the selected positions of the localization elements 24 in physical space and the model space. The computed registration is then used to transform the remaining localization element positions to model space and determine if any markers exist at the projected locations. A brute force iteration is made in groups of 3 as just described. When projecting the remaining points from physical space to model space to test the correlation guess, a test can be performed for the existence of a marker in model space within a settable threshold 3-D distance. If present, a 3-D error can be computed and the correlation resulting in the lowest error can be noted and recorded. This technique discards points in model space that do not have a corresponding point in physical space (i.e., false positives in the list of marker positions determined during segmentation).


Because the number of localization element positions is relatively low, it can be fairly computationally inexpensive to perform the iterative process described above to search all possible correlation combinations. The process is implemented such that the affine transform used to compute rigid body registration between the model space and the physical space for each 3-point correlation is abstract, and the actual implementation can be defined and determined at runtime. It is possible to improve the speed of the process by stopping the solution search iterations if a solution is identified that meets the specified criteria. For example, when computing the error for a correlation guess, the projection loop-and-fail for the correlation guess can be reduced if any single reference point in the physical space fails to map to a point in model space within a specified error threshold. Each potential correlation combination is evaluated by one or more criteria to determine the correlation between segmented markers in model space and physical localization element locations. Examples of evaluation criteria include computing the transformation using three points, and then projecting the remaining physical points to model space as described previously. Other examples include incorporating coil orientation information between the segmented markers and 5- or 6-degrees of freedom (DOF) localization elements, or applying externally available information, such as requiring the user to attach the localization elements in a certain configuration. This correlation technique can account for physical localization elements being in a slightly different relative position than the model space markers since the localization elements process can be performed when the localization elements are anywhere in the periodic cycle of the dynamic body.


After the correlation process, the processor 30 can perform an automatic registration process. The process of registration tracks temporal movement of the dynamic body via the movement of the markers 22, and when temporally valid, computes the transformation between the physical space and the model space.


A measure of a temporal position is referred to herein as a “cost-function.” An automatic registration algorithm uses abstract objects so that the cost-function can be defined/determined at runtime. For example, one possible cost function is the average distance between reference points (e.g., positions of localization elements 24). Cost-functions can compute a temporal measure for a group of reference points independent of the space in which the points are known since the measure is based upon landmark positions relative to each other. Once the correlation is established, the localization element locations in physical space can be periodically evaluated using a cost-function to determine when the dynamic body most closely matches the point in the periodic phase of the first time interval (image acquisition). Examples of cost-functions can include: average distance between markers; max/min axis ratio of bounding ellipsoid; and a ratio between minimum and maximum 3D distance between markers. The cost-function can be, for example, determined in patient model space to ensure that moving the patient and/or localizing machinery will not affect the outcome/solution/computation.


A cost-function can be used to establish a measure of the marker positions within the plurality of images during the first time interval. The same cost-function can then be applied continuously to the correlated localization element positions during the second time interval. When the cost-function indicates that the positions of the localization elements in the second time interval have the same relative positions as the marker positions in the first time interval, then the dynamic body can be identified as being at the same temporal point along the path of motion as the first time interval. During the time that the cost-function indicates that the dynamic body is at the same temporal point along the path of motion as the first time interval, then the automatically correlated markers from the first time interval and localization elements from the second time interval can be used to automatically compute a registration. When the cost-function indicates that the registration is valid, then the position and navigational path of a medical instrument can be displayed on a computer screen superimposed onto images of the dynamic body acquired during the first time interval.


After performing the automated segmentation and correlation processes, a list of position data for the localization elements 24 in image space is obtained. This represents the position of the markers 22, and therefore the position of the dynamic body B when the image dataset was acquired. This information is used as the “temporal reference” for the image dataset and represents the nominal reference point position for the dataset. For multiple images acquired at different points in the patient temporal cycle (e.g., at inspiration and expiration of the respiratory cycle), the segmentation-correlation process can be repeated and a temporal reference position can be determined for each image.


Once the temporal reference is established for each image dataset, a registration filter can be used to compare the position of the localization elements 24 in the physical space to the temporal reference location for each image dataset. If the positions of the localization elements 24 are sufficiently close to the temporal reference for a dataset (i.e., the image dataset), then the dataset can be used for navigation for that temporal moment by computing the affine transformation between the physical space and model space. The transformation is then used to project information such as reformatted images, segmentations, informatics, etc. The threshold that determines how close the physical configuration must be to the locations in the image dataset can be modified at runtime to allow the sensitivity or temporal resolution to be modified.


Through the automatic registration process, the relative marker positions at the time of the 3-D scan can be determined. This acquisition of relative marker position allows the point in the respiratory cycle at which the scan was acquired to be determined and navigation gated to that same point in the cycle during a subsequent medical procedure. The resulting registration is relative to the markers affixed to the patient, which allows the patient to be repositioned relative to the scan gantry, table position, and/or localization machinery without invalidating the registration, as long as the markers remain in a fixed position relative to the patient.


As stated previously, the automatic segmentation-correlation-registration process can be performed using an apparatus that includes a garment, such as a garment disclosed in the '028 application. Such an apparatus can be used with the systems and methods described herein to perform the same automatic-segmentation-registration processes described above, except in such an embodiment, the markers and localization elements are coupled to the patient through the use of a garment. All other devices described with reference to FIGS. 1 and 2 can be used in this embodiment to perform the same automatic segmentation-correlation-registration processes as described above.



FIG. 3 illustrates an apparatus 210 that includes a garment 220 that is tubular shaped and can be constructed with a flexible and/or stretchable material. This particular garment configuration is only one example of a garment that can be used. It should be understood that other garment configurations can alternatively be used, such as those described in the '028 application. The apparatus 210 further includes multiple markers or fiducials 222 coupled to the garment 220 at spaced locations. A plurality of localization elements 224 are removably coupled proximate to the locations of markers 222, such that during a first time interval as described above, images can be taken without the elements 224 being coupled to the garment 220. In other embodiments, the localization elements 224 need not be removably coupled to the markers 222. For example, the localization elements 224 can be fixedly coupled to the garment 220. In addition, the localization elements 224 can be coupled to the garment 220 during the pre-procedure imaging.


The garment 220 can be positioned over a portion of a patient's body (proximate dynamic anatomy), such as around the upper or lower torso of the patient at a fixed location relative to the patient during both a first time period, in which images are taken of the dynamic anatomy (model or image space), and during a second time period, in which a medical procedure is being performed on the dynamic anatomy (physical space). The stretchability of the garment 220 allows the garment 220 to at least partially constrict some of the movement of the portion of the body for which it is coupled. The markers 222 are coupled to the garment 220 at a fixed location on the garment 220, thus the markers 222 are also coupled to the patient at a fixed location relative to the dynamic anatomy during both the first time period and the second time period.



FIG. 4 is a graphical illustration indicating how the apparatus 210 (shown without localization elements 224) can move and change orientation and shape during movement of a dynamic body, such as a mammalian body M. The graph is one example of how the lung volume can change during inhalation (inspiration) and exhalation (expiration) of the mammalian body M. The corresponding changes in shape and orientation of the apparatus 210 during inhalation and exhalation are also illustrated. Although FIG. 4 is being described with reference to an embodiment including a garment, an embodiment that does not include a garment can be similarly described. The six markers 222 shown in FIG. 3 are labeled a, b, c, d, e, and f. As described above, images of the dynamic anatomy with the apparatus 210 coupled thereto can be taken during a first time interval. The images can include an indication of relative position of each of the markers 222, that is the markers 222 are visible in the images, and the position of each marker 222 can then be identified over a period of time. As illustrated, during expiration of the mammalian body M at times indicated as A and C, a distance X between markers a and b is smaller than during inspiration of the mammalian body M, at the time indicated as B. Likewise, a distance Y between markers b and f is greater during inspiration than during expiration.



FIG. 7 is a flowchart illustrating a method according to another embodiment of the invention. A method includes at step 80 receiving during a first time interval image data associated with a plurality of images of a dynamic body. The plurality of images include an indication of a position of a first marker on a garment coupled to the dynamic body and a position of a second marker on the garment coupled to the dynamic body. The first marker is coupled to the garment at a first location and the second marker is coupled to the garment at a second location. The first time interval is associated with a path of motion of the dynamic body. At step 82, data associated with a position of a first localization element relative to the dynamic body is received, and data associated with a position of a second localization element relative to the dynamic body is received during a second time interval after the first time interval. The first localization element is coupled to the garment at the first location, and the second localization element is coupled to the garment at the second location. The second time interval is associated with a path of motion of the dynamic body and the garment is coupled to the dynamic body in a fixed position relative to the dynamic body during both the first time interval and the second time interval.


During the second time interval, an image from the plurality of images associated with a position of the first marker that is substantially the same as the position of the first localization element relative to the dynamic body and a position of the second marker that is substantially the same as the position of the second localization element relative to the dynamic body are automatically identified at step 84. The automatic identification can be based on an appearance of the markers within the identified image. The automatic identification can also include identifying a position of a third localization element, and projecting that position on to the image data set and determining whether a third marker exists in an image from the image data set at the position of the third localization element. The automatic identification can also include correlating a position of the first localization element during the second time interval with the position of the first marker in the plurality of images. At step 86, the path of motion of the dynamic body is automatically registered during the second time interval is automatically registered with the path of motion of the dynamic body during the first time interval. The automatic registering in step 86 can include identifying at least one temporal reference within the plurality of images and identifying whether the at least one temporal reference is associated with at least one of the first marker or the second marker providing a navigational path for a medical instrument to be directed based on the identified image.


At step 88, a navigational path is provided for a medical instrument to be directed based on the identified image. A physician can use the navigational path to guide a medical instrument to the dynamic body while performing a medical procedure on the dynamic body during the second time interval.


CONCLUSION

While various embodiments of the invention have been described above, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of the invention should not be limited by any of the above-described embodiments, but should be defined only in accordance with the following claims and their equivalents.


The previous description of the embodiments is provided to enable any person skilled in the art to make or use the invention. While the invention has been particularly shown and described with reference to embodiments thereof, it will be understood by those skilled in art that various changes in form and details may be made therein without departing from the spirit and scope of the invention. For example, the garment, markers and localization elements can be constructed from any suitable material, and can be a variety of different shapes and sizes, not necessarily specifically illustrated, while still remaining within the scope of the invention.


While a relatively small number of markers are discussed, the system is scalable and the use of any number of markers is contemplated. For example, a garment may include between 2 and 20 markers, 10-50 markers, etc. Additionally, variations in the automated processes can be used to achieve the same, or substantially the same, desired results.

Claims
  • 1. A method for imaging a dynamic body of a patient having predictable movement during a periodic cycle, comprising: a) attaching to the dynamic body: i) a first marker at a first location on the patient,ii) a second marker at a second location on the patient,iii) a third marker at a third location on the patient, andiv) a fourth marker at a fourth location on the patient,wherein the first, second, third, and fourth locations on the patient move relative to each other during the periodic cycle;b) receiving from an imaging device a plurality of 3-D images of the dynamic body over the periodic cycle taken during a first time interval prior to a medical procedure, each image showing a position of the first, second, third, and fourth markers;c) attaching to the dynamic body: i) a first localization element at the first location,ii) a second localization element at the second location,iii) a third localization element at the third location, andiv) a fourth localization element at the fourth location;d) after step c), receiving at a second time during the medical procedure, data identifying positions for the first, second, third, and fourth localization elements; ande) automatically identifying for the second time an identified image from the plurality of images by: i) associating the position of the first, second, and third localization elements with the positions of the first, second, and third markers in the identified image,ii) computing an affine transformation for the associated positions;iii) applying the computed affine transformation to the fourth localization element to compute an expected position for the fourth marker;iv) confirming that the position of the fourth marker in the identified image matches the expected position computed by the affine transformation.
  • 2. The method of claim 1, further comprising: f) displaying the identified image for use in navigating a medical instrument during the medical procedure.
  • 3. The method of claim 1, wherein the process of automatically identifying the identified image comprises iteratively associating the position of the first, second, and third localization elements with a plurality of sets of locations for markers in the plurality of 3-D images.
  • 4. The method of claim 1, wherein the markers are attached to the locations on the patient using adhesive.
  • 5. The method of claim 4, wherein the localization elements are attached at the locations on the patient by physically coupling the localization elements to the markers.
  • 6. The method of claim 1, wherein the imaging device utilizes X-rays.
  • 7. The method of claim 6, wherein the localization elements comprise electro-magnetic coils in which the data identifying positions for the localization elements is derived by measuring voltages induced in the coils.
  • 8. The method of claim 1, wherein the markers are attached to the locations on the patient using straps.
  • 9. The method of claim 1, wherein the markers are attached to the locations on the patient by attaching the markers to a garment that is attached to the patient.
  • 10. The method of claim 1, wherein step c) occurs after the first time interval.
  • 11. The method of claim 1, wherein step c) occurs before the first time interval.
  • 12. The method of claim 1, further comprising: e) automatically registering a path of motion of the dynamic body during a second time interval that includes the second time with the path of motion of the dynamic body during the first time interval based on the automatically identifying for times during the second time interval.
  • 13. The method of claim 1, wherein the images of the dynamic body are 3-D images.
  • 14. The method of claim 1, wherein the localization elements comprise reflective markers.
  • 15. The method of claim 1, wherein the localization elements comprise infrared light emitting diodes.
CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a division of U.S. patent application Ser. No. 14/957,688, filed on Dec. 3, 2015 (the '688 application). The '688 application is a continuation of U.S. application Ser. No. 13/035,945, filed on Feb. 26, 2011, which issued as U.S. Pat. No. 9,218,663 on Dec. 22, 2015 (the '945 application). The '945 application is a divisional application of Ser. No. 11/410,143, filed on Apr. 25, 2006, which issued as U.S. Pat. No. 7,920,909 (the '143 application). The '143 application is a continuation-in-part of U.S. patent application Ser. No. 11/224,028, filed on Sep. 13, 2005. All of the above applications and documents are herein incorporated by reference in their entireties.

US Referenced Citations (405)
Number Name Date Kind
3788324 Lim Jan 1974 A
4421106 Uehara Dec 1983 A
4583538 Onik Apr 1986 A
5053042 Bidwell Oct 1991 A
5081997 Bosley, Jr. Jan 1992 A
5158088 Nelson Oct 1992 A
5186174 Schlondorff Feb 1993 A
5238804 Maskasky Aug 1993 A
5251165 James, III Oct 1993 A
5251635 Dumoulin Oct 1993 A
5265610 Darrow Nov 1993 A
5295493 Radisch, Jr. Mar 1994 A
5348011 Nessaiver Sep 1994 A
5377678 Dumoulin Jan 1995 A
5391199 Ben-haim Feb 1995 A
5437292 Kipshidze Aug 1995 A
5483691 Heck Jan 1996 A
5483961 Kelly Jan 1996 A
5577502 Darrow Nov 1996 A
5581183 Lindstedt Dec 1996 A
5644612 Moorman Jul 1997 A
5671739 Darrow Sep 1997 A
5674498 Inoue Oct 1997 A
5718241 Ben-haim Feb 1998 A
5730129 Darrow Mar 1998 A
5740808 Panescu Apr 1998 A
5765561 Chen Jun 1998 A
5769789 Wang Jun 1998 A
5769861 Vilsmeier Jun 1998 A
5771306 Stork Jun 1998 A
5787886 Kelly Aug 1998 A
5803089 Ferre Sep 1998 A
5814022 Antanavich Sep 1998 A
5814066 Spotnitz Sep 1998 A
5833608 Acker Nov 1998 A
5840025 Ben-haim Nov 1998 A
5868673 Vesely Feb 1999 A
5928248 Acker Jul 1999 A
5951461 Nyo Sep 1999 A
5978696 Vomlehn Nov 1999 A
6016439 Acker Jan 2000 A
6019724 Gronningsaeter Feb 2000 A
6026173 Svenson Feb 2000 A
6078175 Foo Jun 2000 A
6122538 Sliwa, Jr. Sep 2000 A
6122541 Cosman Sep 2000 A
6132396 Antanavich Oct 2000 A
6139508 Simpson Oct 2000 A
6144875 Schweikard Nov 2000 A
6167296 Shahidi Dec 2000 A
6173201 Front Jan 2001 B1
6188355 Gilboa Feb 2001 B1
6198959 Wang Mar 2001 B1
6201987 Dumoulin Mar 2001 B1
6226543 Gilboa May 2001 B1
6226548 Foley May 2001 B1
6233476 Strommer May 2001 B1
6235038 Hunter May 2001 B1
6236875 Bucholz May 2001 B1
6246896 Dumoulin Jun 2001 B1
6246898 Vesely Jun 2001 B1
6253770 Acker Jul 2001 B1
6254550 Mcnamara Jul 2001 B1
6267769 Truwit Jul 2001 B1
D446610 Rosenberg Aug 2001 S
6275560 Blake Aug 2001 B1
6282442 Destefano Aug 2001 B1
6285902 Kienzle, III Sep 2001 B1
6298259 Kucharczyk Oct 2001 B1
6314310 Ben-haim Nov 2001 B1
6314311 Williams Nov 2001 B1
6314312 Wessels Nov 2001 B1
6317616 Glossop Nov 2001 B1
6317619 Boernert Nov 2001 B1
6330356 Sundareswaran Dec 2001 B1
6332089 Acker Dec 2001 B1
6332891 Himes Dec 2001 B1
6335617 Osadchy Jan 2002 B1
6335623 Damadian Jan 2002 B1
6340363 Bolger Jan 2002 B1
6347240 Foley Feb 2002 B1
6348058 Melkent Feb 2002 B1
6351573 Schneider Feb 2002 B1
6351659 Vilsmeier Feb 2002 B1
6361759 Frayne Mar 2002 B1
6362821 Gibson Mar 2002 B1
6368331 Front Apr 2002 B1
6369571 Damadian Apr 2002 B1
6369574 Ederlöv Apr 2002 B1
6373998 Thirion Apr 2002 B2
6379302 Kessman Apr 2002 B1
6380732 Gilboa Apr 2002 B1
6381485 Hunter Apr 2002 B1
6402762 Hunter Jun 2002 B2
6418238 Shiratani Jul 2002 B1
6421551 Kuth Jul 2002 B1
6424856 Vilsmeier Jul 2002 B1
6425865 Salcudean Jul 2002 B1
6430430 Gosche Aug 2002 B1
6434415 Foley Aug 2002 B1
6434507 Clayton Aug 2002 B1
6437571 Danby Aug 2002 B1
6442417 Shahidi Aug 2002 B1
6445186 Damadian Sep 2002 B1
6445943 Ferre Sep 2002 B1
6455182 Silver Sep 2002 B1
6461372 Jensen Oct 2002 B1
6468265 Evans Oct 2002 B1
6469508 Damadian Oct 2002 B1
6470066 Takagi Oct 2002 B2
6470207 Simon Oct 2002 B1
6473635 Rasche Oct 2002 B1
6477400 Barrick Nov 2002 B1
6478793 Cosman Nov 2002 B1
6478802 Kienzle, III Nov 2002 B2
6483948 Spink Nov 2002 B1
6484049 Seeley Nov 2002 B1
6485413 Boppart Nov 2002 B1
D466609 Glossop Dec 2002 S
6490467 Bucholz Dec 2002 B1
6490475 Seeley Dec 2002 B1
6490477 Zylka Dec 2002 B1
6491699 Henderson Dec 2002 B1
6491702 Heilbrun Dec 2002 B2
6493574 Ehnholm Dec 2002 B1
6496007 Damadian Dec 2002 B1
6501981 Schweikard Dec 2002 B1
6504893 Flohr Jan 2003 B1
6504894 Pan Jan 2003 B2
6516213 Nevo Feb 2003 B1
6517485 Torp Feb 2003 B2
6527443 Vilsmeier Mar 2003 B1
6535756 Simon Mar 2003 B1
6538634 Chui Mar 2003 B1
6539127 Roche Mar 2003 B1
6541947 Dittmer Apr 2003 B1
6541973 Danby Apr 2003 B1
6544041 Damadian Apr 2003 B1
6547782 Taylor Apr 2003 B1
6558333 Gilboa May 2003 B2
6562059 Edwards May 2003 B2
6567687 Front May 2003 B2
6580938 Acker Jun 2003 B1
6584174 Schubert Jun 2003 B2
6584339 Galloway, Jr. Jun 2003 B2
6591130 Shahidi Jul 2003 B2
6593884 Gilboa Jul 2003 B1
6606513 Lardo Aug 2003 B2
6609022 Vilsmeier Aug 2003 B2
6615155 Gilboa Sep 2003 B2
6636757 Jascob Oct 2003 B1
6650924 Kuth Nov 2003 B2
6666579 Jensen Dec 2003 B2
6674833 Shahidi Jan 2004 B2
6675032 Chen Jan 2004 B2
6675033 Lardo Jan 2004 B1
6687531 Ferre Feb 2004 B1
6690960 Chen Feb 2004 B2
6694167 Ferre Feb 2004 B1
6697664 Kienzle, III Feb 2004 B2
6702780 Gilboa Mar 2004 B1
6711429 Gilboa Mar 2004 B1
6714629 Vilsmeier Mar 2004 B2
6714810 Grzeszczuk Mar 2004 B2
6725080 Melkent Apr 2004 B2
6738656 Ferre May 2004 B1
6772002 Schmidt Aug 2004 B2
6774624 Anderson Aug 2004 B2
6782287 Grzeszczuk Aug 2004 B2
6796988 Melkent Sep 2004 B2
6799569 Danielsson Oct 2004 B2
6823207 Jensen Nov 2004 B1
6826423 Hardy Nov 2004 B1
6833814 Gilboa Dec 2004 B2
6850794 Shahidi Feb 2005 B2
6856826 Seeley Feb 2005 B2
6856827 Seeley Feb 2005 B2
6892090 Verard May 2005 B2
6898303 Armato, III May 2005 B2
6899672 Chin May 2005 B2
6907281 Grzeszczuk Jun 2005 B2
6920347 Simon Jul 2005 B2
6925200 Wood Aug 2005 B2
6934575 Ferre Aug 2005 B2
6947788 Gilboa Sep 2005 B2
6968224 Kessman Nov 2005 B2
6978166 Foley Dec 2005 B2
6992477 Govari Jan 2006 B2
6996430 Gilboa Feb 2006 B1
7015859 Anderson Mar 2006 B2
7015907 Tek Mar 2006 B2
7035683 Guendel Apr 2006 B2
7050845 Vilsmeier May 2006 B2
7115100 Mcrury Oct 2006 B2
7130700 Gardeski Oct 2006 B2
7139601 Bucholz Nov 2006 B2
7153297 Peterson Dec 2006 B2
7171257 Thomson Jan 2007 B2
7174201 Govari Feb 2007 B2
7233820 Gilboa Jun 2007 B2
7260426 Schweikard Aug 2007 B2
7300428 Ingenito Nov 2007 B2
7339587 Kropfeld Mar 2008 B2
7357807 Donohoe Apr 2008 B2
7366562 Dukesherer Apr 2008 B2
7371067 Anderson May 2008 B2
7398116 Edwards Jul 2008 B2
7481805 Magnusson Jan 2009 B2
7505806 Masutani Mar 2009 B2
7555330 Gilboa Jun 2009 B2
7594925 Danek Sep 2009 B2
7599730 Hunter Oct 2009 B2
7641609 Ohnishi Jan 2010 B2
7659912 Akimoto Feb 2010 B2
7697972 Verard Apr 2010 B2
7756563 Higgins Jul 2010 B2
7889905 Higgins Feb 2011 B2
7901348 Soper Mar 2011 B2
7920909 Lyon Apr 2011 B2
7962193 Edwards Jun 2011 B2
7969143 Gilboa Jun 2011 B2
7985187 Wibowo Jul 2011 B2
7998062 Gilboa Aug 2011 B2
8016749 Clerc Sep 2011 B2
8046052 Verard Oct 2011 B2
8049777 Akimoto Nov 2011 B2
8064669 Higgins Nov 2011 B2
8096957 Conquergood Jan 2012 B2
8102416 Ito Jan 2012 B2
8150138 Ohnishi Apr 2012 B2
8150495 Edwards Apr 2012 B2
8202213 Ito Jun 2012 B2
8214015 Macaulay Jul 2012 B2
8218846 Trumer Jul 2012 B2
8218847 Averbuch Jul 2012 B2
8219179 Ganatra Jul 2012 B2
8317149 Greenburg Nov 2012 B2
8317726 Timberlake Nov 2012 B2
8382662 Soper Feb 2013 B2
8428328 Averbuch Apr 2013 B2
8468003 Gibbs Jun 2013 B2
8473032 Averbuch Jun 2013 B2
8483801 Edwards Jul 2013 B2
8494246 Trumer Jul 2013 B2
8494612 Vetter Jul 2013 B2
8611983 Glossop Dec 2013 B2
8611984 Greenburg Dec 2013 B2
8632461 Glossop Jan 2014 B2
8672836 Higgins Mar 2014 B2
8675935 Higgins Mar 2014 B2
8696548 Gilboa Apr 2014 B2
8696685 Gilboa Apr 2014 B2
8700132 Ganatra Apr 2014 B2
9218663 Lyon Dec 2015 B2
10617332 Edwards Apr 2020 B2
20010007918 Vilsmeier Jul 2001 A1
20010025142 Wessels Sep 2001 A1
20010029333 Shahidi Oct 2001 A1
20010031919 Strommer Oct 2001 A1
20010031985 Gilboa Oct 2001 A1
20010036245 Thomas, III Nov 2001 A1
20010041835 Front Nov 2001 A1
20020044631 Graumann Apr 2002 A1
20020049375 Strommer Apr 2002 A1
20020049378 Grzeszczuk Apr 2002 A1
20020070970 Wood Jun 2002 A1
20020075994 Shahidi Jun 2002 A1
20020077543 Grzeszczuk Jun 2002 A1
20020077544 Shahidi Jun 2002 A1
20020082492 Grzeszczuk Jun 2002 A1
20020085681 Jensen Jul 2002 A1
20020115931 Strauss Aug 2002 A1
20020143317 Glossop Oct 2002 A1
20020161295 Edwards Oct 2002 A1
20030000535 Robert, Jr. Jan 2003 A1
20030004411 Govari Jan 2003 A1
20030016852 Kaufman Jan 2003 A1
20030018251 Solomon Jan 2003 A1
20030023161 Govari Jan 2003 A1
20030028091 Simon Feb 2003 A1
20030029464 Chen Feb 2003 A1
20030032878 Shahidi Feb 2003 A1
20030040667 Feussner Feb 2003 A1
20030051733 Kotmel Mar 2003 A1
20030074011 Gilboa Apr 2003 A1
20030088179 Seeley May 2003 A1
20030125622 Schweikard Jul 2003 A1
20030130576 Seeley Jul 2003 A1
20030139663 Graumann Jul 2003 A1
20030199785 Hibner Oct 2003 A1
20030208116 Liang Nov 2003 A1
20030208122 Melkent Nov 2003 A1
20030216631 Bloch Nov 2003 A1
20030220557 Cleary Nov 2003 A1
20040005088 Jeung Jan 2004 A1
20040006268 Gilboa Jan 2004 A1
20040013548 Seto Jan 2004 A1
20040034300 Verard Feb 2004 A1
20040049121 Yaron Mar 2004 A1
20040076259 Jensen Apr 2004 A1
20040091143 Hu May 2004 A1
20040092815 Schweikard May 2004 A1
20040097805 Verard May 2004 A1
20040097806 Hunter May 2004 A1
20040116803 Jascob Jun 2004 A1
20040122311 Cosman Jun 2004 A1
20040138548 Strommer Jul 2004 A1
20040152970 Hunter Aug 2004 A1
20040152974 Solomon Aug 2004 A1
20040167393 Solar Aug 2004 A1
20040193042 Scampini Sep 2004 A1
20040210125 Chen Oct 2004 A1
20040249267 Gilboa Dec 2004 A1
20050010099 Raabe Jan 2005 A1
20050020900 Yngvesson Jan 2005 A1
20050027186 Chen Feb 2005 A1
20050033149 Strommer Feb 2005 A1
20050038337 Edwards Feb 2005 A1
20050065433 Anderson Mar 2005 A1
20050085718 Shahidi Apr 2005 A1
20050085793 Glossop Apr 2005 A1
20050107679 Geiger May 2005 A1
20050107688 Strommer May 2005 A1
20050113809 Melkent May 2005 A1
20050137661 Sra Jun 2005 A1
20050143651 Verard Jun 2005 A1
20050169510 Zuhars Aug 2005 A1
20050182295 Soper Aug 2005 A1
20050182319 Glossop Aug 2005 A1
20050187482 O'brien Aug 2005 A1
20050197568 Vass Sep 2005 A1
20050203383 Moctezuma Sep 2005 A1
20050234335 Simon Oct 2005 A1
20050288574 Thornton Dec 2005 A1
20050288578 Durlak Dec 2005 A1
20060004281 Saracen Jan 2006 A1
20060025677 Verard Feb 2006 A1
20060045318 Schoisswohl Mar 2006 A1
20060050942 Bertram Mar 2006 A1
20060050988 Kraus Mar 2006 A1
20060058647 Strommer Mar 2006 A1
20060063998 Von Jako Mar 2006 A1
20060064006 Strommer Mar 2006 A1
20060074292 Thomson Apr 2006 A1
20060074299 Sayeh Apr 2006 A1
20060074304 Sayeh Apr 2006 A1
20060079759 Vaillant Apr 2006 A1
20060084867 Tremblay Apr 2006 A1
20060093089 Vertatschitsch May 2006 A1
20060094958 Marquart May 2006 A1
20060106292 Anderson May 2006 A1
20060116634 Shachar Jun 2006 A1
20060122497 Glossop Jun 2006 A1
20060142798 Holman Jun 2006 A1
20060173269 Glossop Aug 2006 A1
20060173291 Glossop Aug 2006 A1
20060184016 Glossop Aug 2006 A1
20060189867 Revie Aug 2006 A1
20060247511 Anderson Nov 2006 A1
20060258933 Ellis Nov 2006 A1
20060270976 Savage Nov 2006 A1
20070032723 Glossop Feb 2007 A1
20070038058 West Feb 2007 A1
20070066887 Mire Mar 2007 A1
20070110289 Fu May 2007 A1
20070129629 Beauregard Jun 2007 A1
20070167714 Kiraly Jul 2007 A1
20070167744 Beauregard Jul 2007 A1
20070225559 Clerc Sep 2007 A1
20070232896 Gilboa Oct 2007 A1
20070244355 Shaw Oct 2007 A1
20070249896 Goldfarb Oct 2007 A1
20070276180 Greenburg Nov 2007 A1
20080071142 Gattani Mar 2008 A1
20080071143 Gattani Mar 2008 A1
20080118135 Averbuch May 2008 A1
20080125760 Gilboa May 2008 A1
20080132757 Tgavalekos Jun 2008 A1
20080140114 Edwards Jun 2008 A1
20080167639 Gilboa Jul 2008 A1
20080221442 Tolkowsky Sep 2008 A1
20080247622 Aylward Oct 2008 A1
20080255416 Gilboa Oct 2008 A1
20080262297 Gilboa Oct 2008 A1
20080262342 Averbruch Oct 2008 A1
20080269561 Banik Oct 2008 A1
20080287803 Li Nov 2008 A1
20090054908 Zand Feb 2009 A1
20090088600 Meloul Apr 2009 A1
20090156895 Higgins Jun 2009 A1
20090156951 Averbuch Jun 2009 A1
20090209817 Averbuch Aug 2009 A1
20090227861 Ganatra Sep 2009 A1
20090240140 Fitelzon Sep 2009 A1
20090240198 Averbuch Sep 2009 A1
20090284255 Zur Nov 2009 A1
20100036241 Mayse Feb 2010 A1
20100041949 Tolkowsky Feb 2010 A1
20100160733 Gilboa Jun 2010 A1
20120046521 Hunter Feb 2012 A1
20120059220 Holsing Mar 2012 A1
20120059248 Holsing Mar 2012 A1
20120065481 Hunter Mar 2012 A1
20120071753 Hunter Mar 2012 A1
20120123296 Hashimshony May 2012 A1
Foreign Referenced Citations (87)
Number Date Country
19751761 Oct 1998 DE
19725137 Jan 1999 DE
19829224 Jan 2000 DE
19909816 May 2000 DE
10000937 Aug 2001 DE
10136709 Feb 2003 DE
10161160 Jun 2003 DE
102005010010 Sep 2005 DE
102004030836 Jan 2006 DE
102005026251 Jan 2006 DE
102005038394 Mar 2006 DE
102005050286 Apr 2006 DE
102004058122 Jul 2006 DE
0501993 Sep 1992 EP
501993 Sep 1992 EP
0869745 Oct 1998 EP
900048 Mar 1999 EP
977510 Feb 2000 EP
1079240 Feb 2001 EP
1152706 Nov 2001 EP
1181897 Feb 2002 EP
1319368 Jun 2003 EP
1374792 Jan 2004 EP
1374793 Jan 2004 EP
1391181 Feb 2004 EP
1421913 May 2004 EP
1464285 Oct 2004 EP
1504713 Feb 2005 EP
1504726 Feb 2005 EP
1519140 Mar 2005 EP
1523951 Apr 2005 EP
1561423 Aug 2005 EP
1629774 Mar 2006 EP
1629789 Mar 2006 EP
2380550 Oct 2011 EP
2876273 Apr 2006 FR
2000023941 Jan 2000 JP
WO9424933 Nov 1994 WO
WO9501757 Jan 1995 WO
WO9608209 Mar 1996 WO
WO9610949 Apr 1996 WO
WO9729699 Aug 1997 WO
WO9729709 Aug 1997 WO
WO9836684 Aug 1998 WO
WO9916352 Apr 1999 WO
WO9927839 Jun 1999 WO
WO2000047103 Aug 2000 WO
WO2000049958 Aug 2000 WO
WO2000057767 Aug 2000 WO
WO2000069335 Nov 2000 WO
WO2001001845 Jan 2001 WO
WO2001037748 May 2001 WO
WO2001062134 Aug 2001 WO
WO2001076496 Oct 2001 WO
WO2001076497 Oct 2001 WO
WO2001087136 Nov 2001 WO
WO2001093745 Dec 2001 WO
WO2002000093 Jan 2002 WO
WO2002000103 Jan 2002 WO
WO2002019936 Mar 2002 WO
WO2002022015 Mar 2002 WO
WO2002024051 Mar 2002 WO
WO2002056770 Jul 2002 WO
WO2002064011 Aug 2002 WO
WO2002082375 Oct 2002 WO
WO2002098273 Dec 2002 WO
WO2004046754 Jun 2004 WO
WO2004060157 Jul 2004 WO
WO2004062497 Jul 2004 WO
WO2005016166 Feb 2005 WO
WO2005070318 Aug 2005 WO
WO2005077293 Oct 2005 WO
WO2005101277 Oct 2005 WO
WO2005111942 Nov 2005 WO
WO2006002396 Jan 2006 WO
WO2006005021 Jan 2006 WO
WO2006027781 Mar 2006 WO
2006039009 Apr 2006 WO
WO2006051523 May 2006 WO
WO2006090141 Aug 2006 WO
WO2007002079 Jan 2007 WO
WO2007031314 Mar 2007 WO
WO2007033206 Mar 2007 WO
WO2007062051 May 2007 WO
WO2007084893 Jul 2007 WO
WO2009158578 Dec 2009 WO
WO2012024686 Feb 2012 WO
Non-Patent Literature Citations (7)
Entry
European Patent Office, Extended Search Report issued for EP 11818898.6, 6 pages dated Dec. 20, 2013.
Patent Cooperation Treaty, International Search Report and Written Opinion from PCT/US06/35548, dated Aug. 20, 2007, 7 pages.
Oct. 3, 2018 USPTO Office Action (U.S. Appl. No. 14/957,719).
Jan. 25, 2019 USPTO Office Action (U.S. Appl. No. 14/957,688).
Jan. 25, 2019 USPTO Office Action (U.S. Appl. No. 14/957,719).
Patent Cooperation Treaty, International Search Report issued for PCT/US2011/048669, dated Apr. 9, 2012, 7 pages Apr. 9, 2012.
May 14, 2019 USPTO Office Action (U.S. Appl. No. 14/957,688).
Related Publications (1)
Number Date Country
20200015714 A1 Jan 2020 US
Divisions (2)
Number Date Country
Parent 14957688 Dec 2015 US
Child 16586173 US
Parent 11410143 Apr 2006 US
Child 13035945 US
Continuations (1)
Number Date Country
Parent 13035945 Feb 2011 US
Child 14957688 US
Continuation in Parts (1)
Number Date Country
Parent 11224028 Sep 2005 US
Child 11410143 US