Method and apparatus for perspective inversion

Information

  • Patent Grant
  • 6947786
  • Patent Number
    6,947,786
  • Date Filed
    Thursday, February 28, 2002
    22 years ago
  • Date Issued
    Tuesday, September 20, 2005
    18 years ago
Abstract
A surgical instrument navigation system is provided that allows a surgeon to invert the three-dimensional perspective of the instrument to match their perspective of the actual instrument. The surgical instrument navigation system includes: a surgical instrument; an imaging device that is operable to capture image data representative of a patient; a tracking subsystem that is operable to capture in real-time position data indicative of the position of the surgical instrument; and a data processor adapted to receive the image data from the imaging device and the position data from the tracking subsystem. The data processor is operable to generate a three-dimensional representation of the surgical instrument as it would visually appear from either of at least two different perspectives and to overlay the representation of the surgical instrument onto the image data of the patient. The navigation system further includes a display that is operable to display the representation of the surgical instrument superimposed onto the image data of the patient.
Description
FIELD OF THE INVENTION

The present invention relates generally to surgical instrument navigation systems and, more particularly, to a navigation system that provides perspective inversion of the surgical instrument.


BACKGROUND OF THE INVENTION

Modern diagnostic medicine has benefited significantly from radiology. Radiation, such as x-rays, may be used to generate images of internal body structures. In general, radiation is emanated towards a patient's body and absorbed in varying amounts by tissues in the body. An x-ray image is then created based on the relative differences of detected radiation passing through the patients' body.


Surgical navigation guidance can provide a tool for helping the physician perform surgery. One known technique involves tracking position in real-time of a surgical instrument in the patient's anatomy as it is represented by an x-ray image. The virtual representation of the surgical instrument is a three-dimensional object superimposed onto the two-dimensional image of the patient. Thus, the three-dimensional representation appears to be directed into or out of the two-dimensional image of the patient. An exemplary surgical navigation guidance system is disclosed in U.S. application Ser. No. 09/274,972 filed on Mar. 23, 1999 which is assigned to the assignee of the present invention and incorporated herein by reference.


When an image is acquired, it is acquired from a certain perspective or point-of-view. In the case of a C-arm imaging device, the perspective is determined by the orientation of the C-Arm around the patient. Specifically, the perspective is along the line connecting the image source and the image receiver. If the surgeon navigates the surgical instrument from the position of the image receiver, the perspective of the virtual representation of the instrument will match the surgeon's perspective of the actual instrument. However, if the surgeon navigates from the position of the radiation source, the perspective of the virtual representation of the instrument will appear “flipped” from the surgeon's perspective of the actual instrument.


Therefore, it is desirable to provide a surgical navigation system that allows the surgeon to invert or “flip” the three-dimensional perspective of the instrument to match their perspective of the actual instrument.


SUMMARY OF THE INVENTION

In accordance with the present invention, a surgical instrument navigation system is provided that allows a surgeon to invert the three-dimensional perspective of the instrument to match their perspective of the actual instrument. The surgical instrument navigation system includes: a surgical instrument; an imaging device that is operable to capture image data representative of a patient; a tracking subsystem that is operable to capture in real-time position data indicative of the position of the surgical instrument; and a data processor adapted to receive the image data from the imaging device and the position data from the tracking subsystem. The data processor is operable to generate a three-dimensional representation of the surgical instrument as it would visually appear from either of at least two different perspectives and to overlay the representation of the surgical instrument onto the image data of the patient. The navigation system further includes a display that is operable to display the representation of the surgical instrument superimposed onto the image data of the patient.


For a more complete understanding of the invention, reference may be had to the following specification and to the accompanying drawings.





BRIEF DESCRIPTION OF THE DRAWINGS


FIG. 1 is a diagram of a surgical instrument navigation system in accordance with present invention;



FIG. 2 is a diagram of an ideal and distorted image that may be captured by the surgical navigation system;



FIGS. 3A and 3B illustrates the projective transformation process employed by the surgical navigation system;



FIG. 4 is a flowchart depicting the operation of the surgical navigation system;



FIGS. 5A and 5B illustrate graphical representations of the surgical instrument superimposed onto a two-dimensional image of the patient;



FIG. 6 illustrates an exemplary graphical user interface of the surgical instrument navigation system; and



FIG. 7 is a flowchart depicting how perspective inversion is incorporated into the operation of the surgical instrument navigation system in accordance with the present invention.





DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS


FIG. 1 is a diagram of an exemplary surgical instrument navigation system. The primary component of the surgical instrument navigation system is a fluoroscopic imaging device 100. The fluoroscopic imaging device 100 generally includes a C-arm 103, x-ray source 104, x-ray receiving section 105, a calibration and tracking target 106, and radiation sensors 107. Calibration and tracking target 106 includes infrared reflectors (or alternatively infrared emitters) 109 and calibration markers 111. C-arm control computer 115 allows a physician to control the operation of imaging device 100, such as setting imaging parameters. One appropriate implementation of imaging device 100 is the “Series 9600 Mobile Digital Imaging System,” from OEC Medical Systems, Inc., of Salt Lake City, Utah. It should be noted that calibration and tracking target 106 and radiation sensors 107 are typically not included in the Series 9600 Mobile Digital Imaging System; otherwise the “Series 9600 Mobile Digital Imaging System” is similar to imaging system 100.


In operation, x-ray source 104 generates x-rays that propagate through patient 110 and calibration target 106, and into x-ray receiving section 105. Receiving section 105 generates an image representing the intensities of the received x-rays. Typically, receiving section 105 comprises an image intensifier that converts the x-rays to visible light and a charge coupled device (CCD) video camera that converts the visible light to digital images. Receiving section 105 may also be a device that converts x-rays directly to digital images, thus potentially avoiding distortion introduced by first converting to visible light.


Fluoroscopic images taken by imaging device 100 are transmitted to computer 115, where they may further be forwarded to computer 120. Computer 120 provides facilities for displaying (on monitor 121), saving, digitally manipulating, or printing a hard copy of the received images. Three-dimensional images, such as pre-acquired patient specific CT/MR data set 124 or a three-dimensional atlas data set 126 may also be manipulated by computer 120 and displayed by monitor 121. Images, instead of or in addition to being displayed on monitor 121, may also be displayed to the physician through a heads-up-display.


Although computers 115 and 120 are shown as two separate computers, they alternatively could be variously implemented as multiple computers or as a single computer that performs the functions performed by computers 115 and 120. In this case, the single computer would receive input from both C-arm imager 100 and tracking sensor 130.


Radiation sensors 107 sense the presence of radiation, which is used to determine whether or not imaging device 100 is actively imaging. The result of their detection is transmitted to processing computer 120. Alternatively, a person may manually indicate when device 100 is actively imaging or this function can be built into x-ray source 104, x-ray receiving section 105, or control computer 115.


In operation, the patient is positioned between the x-ray source 104 and the x-ray receiving section 105. In response to an operator's command input at control computer 115, x-rays emanate from source 104 and pass through patient 110, calibration target 106, and into receiving section 105 which generates a two-dimensional image of the patient.


C-arm 103 is capable of rotating relative to patient 110, thereby allowing images of patient 110 to be taken from multiple directions. For example, the physician may rotate C-arm 103 in the direction of arrows 108 or about the long axis of the patient. Each of these directions of movement involves rotation about a mechanical axis of the C-arm. In this example, the long axis of the patient is aligned with the mechanical axis of the C-arm.


Raw images generated by receiving section 105 tend to suffer from undesirable distortion caused by a number of factors, including inherent image distortion in the image intensifier and external electromagnetic fields. Drawings representing ideal and distorted images are shown in FIG. 2. Checkerboard 202 represents the ideal image of a checkerboard shaped object. The image taken by receiving section 105, however, can suffer significant distortion, as illustrated by distorted image 204.


The image formation process in a system such as fluoroscopic C-arm imager 100 is governed by a geometric projective transformation which maps lines in the fluoroscope's field of view to points in the image (i.e., within the x-ray receiving section 105). This concept is illustrated in FIGS. 3A and 3B. Image 300 (and any image generated by the fluoroscope) is composed of discrete picture elements (pixels), an example of which is labeled as 302. Every pixel within image 300 has a corresponding three-dimensional line in the fluoroscope's field of view. For example, the line corresponding to pixel 302 is labeled as 304. The complete mapping between image pixels and corresponding lines governs projection of objects within the field of view into the image. The intensity value at pixel 302 is determined by the densities of the object elements (i.e., portions of a patient's anatomy, operating room table, etc.) intersected by the line 304. For the purposes of computer assisted navigational guidance, it is necessary to estimate the projective transformation which maps lines in the field of view to pixels in the image, and vice versa. Geometric projective transformation is well known in the art.


Intrinsic calibration, which is the process of correcting image distortion in a received image and establishing the projective transformation for that image, involves placing “calibration markers” in the path of the x-ray, where a calibration marker is an object opaque or semi-opaque to x-rays. Calibration markers 111 are rigidly arranged in predetermined patterns in one or more planes in the path of the x-rays and are visible in the recorded images. Tracking targets, such as emitters or reflectors 109, are fixed in a known position relative to calibration markers 111.


Because the true relative position of the calibration markers 111 in the recorded images are known, computer 120 is able to calculate an amount of distortion at each pixel in the image (where a pixel is a single point in the image). Accordingly, computer 120 can digitally compensate for the distortion in the image and generate a distortion-free, or at least a distortion improved image. Alternatively, distortion may be left in the image, and subsequent operations on the image, such as superimposing an iconic representation of a surgical instrument on the image (described in more detail below), may be distorted to match the image distortion determined by the calibration markers. The calibration markers can also be used to estimate the geometric perspective transformation, since the position of these markers are known with respect to the tracking target emitters or reflectors 109 and ultimately with respect to tracking sensor 130. A more detailed explanation of methods for performing intrinsic calibration is described in the references B. Schuele et al., “Correction of Image Intensifier Distortion for Three-Dimensional Reconstruction,” presented at SPIE Medical Imaging 1995, San Diego, Calif., 1995 and G. Champleboux et al., “Accurate Calibration of Cameras and Range Imaging Sensors: the NPBS Method,” Proceedings of the 1992 IEEE International Conference on Robotics and Automation, Nice, France, May 1992, and U.S. application Ser. No. 09/106,109, filed on Jun. 29, 1998 by the present assignee, the contents of which are hereby incorporated by reference.


Calibration and tracking target 106 may be attached to x-ray receiving section 105 of the C-arm. Alternately, the target 106 can be mechanically independent of the C-arm, in which case it should be positioned such that the included calibration markers 111 are visible in each fluoroscopic image to be used in navigational guidance. Element 106 serves two functions. The first, as described above, is holding calibration markers 111 used in intrinsic calibration. The second function, which is described in more detail below, is holding infrared emitters or reflectors 109, which act as a tracking target for tracking sensor 130.


Tracking sensor 130 is a real-time infrared tracking sensor linked to computer 120. Specially constructed surgical instruments and other markers in the field of tracking sensor 130 can be detected and located in three-dimensional space. For example, a surgical instrument 140, such as a drill, is embedded with infrared emitters or reflectors 141 on its handle. Tracking sensor 130 detects the presence and location of infrared emitters or reflectors 141. Because the relative spatial locations of the emitters or reflectors in instrument 140 are known a priori, tracking sensor 130 and computer 120 are able to locate instrument 140 in three-dimensional space using well known mathematical transformations. Instead of using infrared tracking sensor 130 and corresponding infrared emitters or reflectors, other types of positional location devices which are known in the art may be used. For example, positional location devices based on magnetic fields, sonic emissions, or radio waves are also within the scope of the present invention.


Reference frame marker 150, like surgical instrument 140, is embedded with infrared emitters or reflectors, labeled 151. As with instrument 140, tracking sensor 130 similarly detects the spatial location of emitters/reflectors 151, through which tracking sensor 130 and computer 120 determine the three-dimensional position of dynamic reference frame marker 150. The determination of the three-dimensional position of an object relative to a patient is known in the art, and is discussed, for example, in the following references, each of which is hereby incorporated by reference: PCT Publication WO 96/11624 to Bucholz et al., published Apr. 25, 1996; U.S. Pat. No. 5,384,454 to Bucholz; U.S. Pat. No. 5,851,183 to Bucholz; and U.S. Pat. No. 5,871,445 to Bucholz.


During an operation, dynamic reference frame marker 150 is attached in a fixed position relative to the portion of the patient to be operated on. For example, when inserting a screw into the spine of patient 110, dynamic reference frame marker 150 may be physically attached to a portion of the spine of the patient. Because dynamic reference frame 150 is in a fixed position relative to the patient anatomy, and instrument 140 can be accurately located in three dimensional space relative to dynamic reference frame 150, instrument 140 can also be located relative to the patient's anatomy.


As discussed above, calibration and tracking target 106 also includes infrared emitters or reflectors 109 similar to those in instrument 140 or dynamic reference frame 150. Accordingly, tracking sensor 130 and computer 120 may determine the three-dimensional position of calibration target 106 relative to instrument 140 and/or dynamic reference frame 150 and thus the patient position.


In general, the imaging system assists physicians performing surgery by displaying real-time or pre-acquired images, such as fluoroscopic x-ray images, of the patient 110 on display 121. Representations of surgical instruments 140 are overlaid on pre-acquired fluoroscopic images of patient 110 based on the position of the instruments determined by tracking sensor 130. In this manner, the physician is able to see the location of the instrument relative to the patient's anatomy, without the need to acquire real-time fluoroscopic images, thereby greatly reducing radiation exposure to the patient and to the surgical team. “Pre-acquired,” as used herein, is not intended to imply any required minimum duration between receipt of the x-ray signals and displaying the corresponding image. Momentarily storing the corresponding digital signal in computer memory while displaying the fluoroscopic image constitutes pre-acquiring the image.



FIG. 4 is a flowchart depicting the operation of the surgical navigation system. The physician begins by acquiring one or more fluoroscopic x-ray images of patient 110 using imager 100 (step 400). As previously mentioned, acquiring an x-ray image triggers radiation sensors 107, which informs computer 120 of the beginning and end of the radiation cycle used to generate the image. For a fluoroscopic x-ray image acquired with imager 100 to be useable for navigational guidance, imager 100, when acquiring the image, should be stationary with respect to patient 110. If C-arm 103 or patient 110 is moving during image acquisition, the position of the fluoroscope will not be accurately determined relative to the patient's reference frame. Thus, it is important that the recorded position of imager 100 reflects the true position of the imager at the time of image acquisition. If imager 100 moves during the image acquisition process, or if imager 100 moves after image acquisition but before its position is recorded, the calibration will be erroneous, thereby resulting in incorrect graphical overlays. To prevent this type of erroneous image, computer 120 may examine the position information from tracking sensor 130 while radiation sensors 107 are signaling radiation detection. If the calibration and tracking target 106 moves relative to dynamic reference frame 150 during image acquisition, this image is marked as erroneous (Steps 401 and 402).


At the end of the radiation cycle, computer 120 retrieves the acquired image from C-arm control computer 115 and retrieves the location information of target marker 106 and dynamic reference frame 150 from tracking sensor 130. Computer 120 calibrates the acquired image, as described above, to learn its projective transformation and optionally to correct distortion in the image, (step 403), and then stores the image along with its positional information (step 404). The process of steps 400-404 is repeated for each image that is to be acquired (step 405).


Because the acquired images are stored with the positional information of the calibration and tracking target 106 and dynamic reference frame 150, the position of C-arm 103, x-ray source 104, and receiving section 105 for each image, relative to patient 110, can be computed based upon the projective transformation identified in the calibration process. During surgery, tracking sensor 130 and computer 120 detect the position of instrument 140 relative to dynamic reference frame 150, and hence relative to patient 110. With this information, computer 120 dynamically calculates, in real-time, the projection of instrument 140 into each fluoroscopic image as the instrument is moved by the physician. A graphical representation of instrument 140 may then be overlaid on the fluoroscopic images (step 406). The graphical representation of instrument 140 is an iconic representation of where the actual surgical instrument would appear within the acquired fluoroscopic x-ray image if imager 100 was continuously acquiring new images from the same view as the original image. There is no theoretical limit to the number of fluoroscopic images on which the graphical representations of instrument 140 may be simultaneously overlaid.


The graphical representation of the surgical instrument is a three-dimensional object superimposed onto a two-dimensional image of the patient. The three-dimensional representation of the instrument may appear to be directed into or out of the two-dimensional image as shown in FIGS. 5A and 5B. In FIG. 5A, the tip 502 of the instrument 504 and the projected length appear to be directed into the image. Conversely, in FIG. 5B, the tip 502 of the instrument 504 and the projected length appear to be coming out of the image.


When an image is acquired, it is acquired from a certain perspective or point-of-view. In the case of a C-arm imaging device 100, the perspective is determined by the orientation of the C-Arm 103 around the patient 110. Specifically, the perspective is along the line connecting the image source 104 and the image receiver section 105. If the surgeon navigates the surgical instrument from the position of the image receiver section 105, the perspective of the virtual representation of the instrument will match the surgeon's perspective of the actual instrument. However, if the surgeon navigates from the position of the image source 104, the perspective of the virtual representation of the instrument will appear “flipped” from the surgeon's perspective of the actual instrument.


In accordance with the present invention, the surgical instrument navigation system described above has been enhanced to allow a surgeon to invert the graphical representation of the instrument to match their perspective of the actual instrument. In a preferred embodiment, the navigation system provides two possible perspectives: positive (+) or negative (−). The positive state renders the instrument from the perspective of the image receiver section 105; whereas the negative state renders the instrument from the perspective of the image source 104. It is envisioned that either state may be designated the default state. It is further envisioned that more than two perspectives may be available for selection by the surgeon.


Referring to FIG. 6, the perspective of the instrument is selectable using a touch screen operable button 601 provided on the graphical user interface of the navigation system. One skilled in the art will readily recognize that rendering a particular perspective of the instrument does not affect the profile of the instrument or the location of the instrument on the image. The perspective selection only affects the internal contours that give the instrument the appearance into or out of the image as shown in FIGS. 5A and 5B. Although a touch screen operable button is presently preferred, it is envisioned that other techniques for selecting the perspective of the instrument, such as a foot pedal or other switching device in close proximity to the surgeon, are also within the scope of the present invention.


A more detailed description of how perspective inversion is incorporated into the operation of the surgical instrument navigation system is provided in conjunction with FIG. 7. As noted above, the projection of the instrument into the fluoroscopic image is calculated in real-time as the instrument is moved by the surgeon.


To do so, the tracking sensor 130, in conjunction with the computer 120, detects the position of the instrument 140 at step 702 relative to the dynamic reference frame 150, and thus relative to the patient 110. The tracking sensor 130, in conjunction with the computer 120, also determines the position of the tracking target 106 at step 704 relative to the dynamic reference frame 150. Based this position data, the computer 120 can determine the position of the instrument 140 relative to the tracking target 106 at step 706, and calibrate the position of the instrument relative to the image plane of the fluoroscopic images at step 708.


Prior to rendering the image, the navigation system accounts for the various user settings 714, including instrument perspective. The selected perspective setting 714 is input into the computer 120 at step 710 which in turn provides corresponding input to the graphic rendering software. One skilled in the art will readily recognize that other user settings (e.g., zoom, rotate, etc.) may be accounted for by the navigation system.


Lastly, the fluoroscopic image is rendered by the navigation system at step 712. Specifically, the three-dimensional representation of the surgical instrument is rendered from the perspective input by an operator of the navigation system. The representation of the instrument is then superimposed over the previously calibrated image data for the patient. In this way, the perspective of the displayed instrument matches the surgeon's perspective of the actual instrument. As noted above, the representation of the surgical instrument is tracked in real-time as it is moved by the surgeon.


While the invention has been described in its presently preferred form, it will be understood that the invention is capable of modification without departing from the spirit of the invention as set forth in the appended claims.

Claims
  • 1. A surgical instrument navigation system, comprising: a surgical instrument; an imaging device operable to capture image data representative of a patient in a first orientation; a tracking subsystem operable to capture in real-time position data indicative of the position of the surgical instrument; a data processor adapted to receive image data from the imaging device and position data from the tracking subsystem, the data processor being operable to generate a three-dimensional representation of the surgical instrument as it would visually appear from either of at least two different perspectives and to overlay the representation of the surgical instrument onto the image data of the patient in the first orientation; and a display in data communication with the data processor, the display being operable to display the representation of the surgical instrument in at least one of the two different perspectives superimposed onto the image data of the patient in the first orientation.
  • 2. The surgical instrument navigation system of claim 1 wherein the processor is further operable to track in real-time the position of the surgical instrument as it is moved by a surgeon and generate a corresponding representation of the surgical instrument in relation to the image data of the patient.
  • 3. The surgical instrument navigation system of claim 1 wherein the imaging device is further defined as a rotatable C-arm, such that at least one of the two different perspectives is determined by an orientation of the C-arm in relation to the patient.
  • 4. The surgical instrument navigation system of claim 1 wherein the imaging device further includes an image source emanating radiation towards the patient and an image receiver positioned to receive radiation from the image source, such that the generated representation of the surgical instrument visually appears as it would from either the perspective of the image source or the perspective of the image receiver.
  • 5. The surgical instrument navigation system of claim 1 wherein the generated representation of the surgical instrument visually appears from a perspective that is selectable by a user of the navigation system.
  • 6. The surgical instrument navigation system of claim 5 wherein the display further includes a touch screen activated button operable to select the perspective of the generated representation of the surgical instrument.
  • 7. The surgical instrument navigation system of claim 1 wherein the display visually indicates the perspective of the generated representation of the surgical instrument as it appears to a user of the navigation system.
  • 8. The surgical instrument navigation system of claim 1 wherein the imaging device is selected from the group consisting of x-ray imaging device, computed tomography imaging device and magnetic resonance imaging device.
  • 9. The surgical instrument navigation system of claim 1 wherein the tracking subsystem employs a non-contact positional location technique that is based on at least one of radio waves, infrared waves, magnetic fields, or sonic emissions.
  • 10. The surgical instrument navigation system of claim 1, wherein the image data representative of a patient in a first orientation is selected from a group comprising a lateral view, a medial view, and an axial view.
  • 11. The surgical instrument navigation system of claim 1, wherein the display of the representation of the surgical instrument in at least one of the two different perspectives may be superimposed onto the image data of the patient in a single position to represent a perspective of the surgical instrument in the same image data.
  • 12. The surgical instrument navigation system of claim 1, wherein the representation of the surgical instrument in at least one of the two different perspectives includes a first perspective is about 180 degrees about a single axis relative to the second perspective.
  • 13. The surgical instrument navigation system of claim 1, wherein the representation of the surgical instrument includes a three dimensional icon of the surgical instrument that may visually appear from either of at least two different perspectives.
  • 14. A surgical instrument navigation system, comprising: a surgical instrument; an imaging device operable to capture image data representative of a patient in an orientation, the imaging device including an image source emanating radiation towards the patient and an image receiver positioned to receive radiation from the image source; a tracking subsystem operable to capture in real-time position data indicative of the position of the surgical instrument; a data processor adapted to receive image data from the imaging device and position data from the tracking subsystem, the data processor being operable to generate a three-dimensional representation of the surgical instrument as it would visually appear from either the perspective of the image source or the perspective of the image receiver and to overlay the representation of the surgical instrument onto the image data of the patient in the orientation; and a display in data communication with the data processor, the display being operable to display the representation of the surgical instrument superimposed onto the image data of the patient.
  • 15. The surgical instrument navigation system of claim 14 wherein the processor is further operable to track in real-time the position of the surgical instrument as it is moved by a surgeon and generate a corresponding representation of the surgical instrument in relation to the image data of the patient.
  • 16. The surgical instrument navigation system of claim 14 wherein the generated representation of the surgical instrument visually appears from a perspective that is selectable by a user of the navigation system.
  • 17. The surgical instrument navigation system of claim 16 wherein the display is further defined as a graphical user interface having a touch screen activated button to select the perspective of the generated representation of the surgical instrument.
  • 18. The surgical instrument navigation system of claim 14 wherein the imaging device is selected from the group consisting of x-ray imaging device, computed tomography imaging device and magnetic resonance imaging device.
  • 19. The surgical instrument navigation system of claim 14 wherein the tracking subsystem employs a non-contact positional location technique that is based on at least one of radio waves, infrared waves, magnetic fields, or sonic emissions.
  • 20. A method for displaying a virtual representation of a surgical instrument using a surgical instrument navigation system, comprising: capturing image data representative of a patient using an imaging device; rendering a three-dimensional representation of the surgical instrument as it would visually appear from a first perspective; displaying the representation of the surgical instrument superimposed onto the image data of the patient; and subsequently displaying the representation of the surgical instrument as it would visually appear from a second perspective that is selectable by an operator of the surgical instrument navigation system.
  • 21. The method of claim 20 further comprises capturing in real-time position data indicative of the position of the surgical instrument and displaying the representation of the surgical instrument superimposed onto the image data of the patient in accordance with the position data for the surgical instrument.
  • 22. The method of claim 20 wherein the step of subsequently displaying the representation of the surgical instrument further comprises selecting a perspective from which to view the representation of the surgical instrument; rendering a representation of the surgical instrument in accordance with the selected perspective; and displaying the representation of the surgical instrument superimposed onto the image data of the patient.
  • 23. A method for displaying a virtual representation of a surgical instrument using a surgical instrument navigation system, comprising: capturing image data representative of an orientation of at least a portion of a patient using an imaging device; rendering a three-dimensional representation of the surgical instrument as it would visually appear from at least two perspectives relative to the portion of the patient in the orientation; and displaying the representation of the surgical instrument in one of the at least two perspectives superimposed onto the image data of the patient in the orientation.
  • 24. The method of claim 23, further comprising: subsequent to displaying the representation of the surgical instrument in one of the at least two perspectives displaying the representation of the surgical instrument as it would visually appear from a second perspective that is selectable by an operator of the surgical instrument navigation system.
US Referenced Citations (408)
Number Name Date Kind
1576781 Philips Mar 1926 A
1735726 Bornhardt Nov 1929 A
2407845 Nemeyer Sep 1946 A
2650588 Drew Sep 1953 A
2697433 Sehnder Dec 1954 A
3016899 Stenvall Jan 1962 A
3017887 Heyer Jan 1962 A
3061936 Dobbeleer Nov 1962 A
3073310 Mocarski Jan 1963 A
3294083 Alderson Dec 1966 A
3367326 Frazier Feb 1968 A
3439256 Kähne et al. Apr 1969 A
3577160 White May 1971 A
3674014 Tillander Jul 1972 A
3702935 Carey et al. Nov 1972 A
3704707 Halloran Dec 1972 A
3868565 Kiipers Feb 1975 A
3941127 Froning Mar 1976 A
4037592 Kronner Jul 1977 A
4052620 Brunnett Oct 1977 A
4054881 Raab Oct 1977 A
4117337 Staats Sep 1978 A
4173228 Van Steenwyk et al. Nov 1979 A
4202349 Jones May 1980 A
4262306 Renner Apr 1981 A
4287809 Egli et al. Sep 1981 A
4314251 Raab Feb 1982 A
4317078 Weed et al. Feb 1982 A
4328813 Ray May 1982 A
4339953 Iwasaki Jul 1982 A
4358856 Stivender et al. Nov 1982 A
4368536 Pfeiler Jan 1983 A
4396885 Constant Aug 1983 A
4403321 DiMarco Sep 1983 A
4418422 Richter et al. Nov 1983 A
4422041 Lienau Dec 1983 A
4431005 McCormick Feb 1984 A
4485815 Amplatz Dec 1984 A
4543959 Sepponen Oct 1985 A
4548208 Niemi Oct 1985 A
4572198 Codrington Feb 1986 A
4582995 Lim et al. Apr 1986 A
4584577 Temple Apr 1986 A
4613866 Blood Sep 1986 A
4618978 Cosman Oct 1986 A
4621257 Brown Nov 1986 A
4621628 Bludermann Nov 1986 A
4625718 Olerud et al. Dec 1986 A
4642786 Hansen Feb 1987 A
4645343 Stockdale et al. Feb 1987 A
4649504 Krouglicof et al. Mar 1987 A
4651732 Frederick Mar 1987 A
4653509 Oloff et al. Mar 1987 A
4673352 Hansen Jun 1987 A
4706665 Gouda Nov 1987 A
4719419 Dawley Jan 1988 A
4722056 Roberts et al. Jan 1988 A
4722336 Kim et al. Feb 1988 A
4727565 Ericson Feb 1988 A
4737794 Jones Apr 1988 A
4737921 Goldwasser et al. Apr 1988 A
4750487 Zanetti Jun 1988 A
4771787 Wurster et al. Sep 1988 A
4791934 Brunnett Dec 1988 A
4793355 Crum et al. Dec 1988 A
4797907 Anderton Jan 1989 A
4803976 Frigg et al. Feb 1989 A
4821206 Arora Apr 1989 A
4821731 Martinelli et al. Apr 1989 A
4836778 Baumrind et al. Jun 1989 A
4845771 Wislocki et al. Jul 1989 A
4849692 Blood Jul 1989 A
4862893 Martinelli Sep 1989 A
4889526 Rauscher et al. Dec 1989 A
4905698 Strohl, Jr. et al. Mar 1990 A
4923459 Nambu May 1990 A
4931056 Ghajar et al. Jun 1990 A
4945305 Blood Jul 1990 A
4945914 Allen Aug 1990 A
4951653 Fry et al. Aug 1990 A
4977655 Martinelli Dec 1990 A
4989608 Ratner Feb 1991 A
4991579 Allen Feb 1991 A
5002058 Martinelli Mar 1991 A
5005592 Cartmell Apr 1991 A
5013317 Cole et al. May 1991 A
5016639 Allen May 1991 A
5027818 Bova et al. Jul 1991 A
5030196 Inoue Jul 1991 A
5030222 Calandruccio et al. Jul 1991 A
5031203 Trecha Jul 1991 A
5042486 Pfeiler et al. Aug 1991 A
5050608 Watanabe et al. Sep 1991 A
5054492 Scribner et al. Oct 1991 A
5057095 Fabian Oct 1991 A
5059789 Salcudean Oct 1991 A
5079699 Tuy et al. Jan 1992 A
5086401 Glassman et al. Feb 1992 A
5094241 Allen Mar 1992 A
5097839 Allen Mar 1992 A
5099845 Besz et al. Mar 1992 A
5105829 Fabian et al. Apr 1992 A
5107839 Houdek et al. Apr 1992 A
5107843 Aarnio et al. Apr 1992 A
5107862 Fabian et al. Apr 1992 A
5109194 Cantaloube Apr 1992 A
5119817 Allen Jun 1992 A
5142930 Allen et al. Sep 1992 A
5152288 Hoenig et al. Oct 1992 A
5160337 Cosman Nov 1992 A
5161536 Vilkomerson et al. Nov 1992 A
5178164 Allen Jan 1993 A
5178621 Cook et al. Jan 1993 A
5186174 Schlondorff et al. Feb 1993 A
5187475 Wagener et al. Feb 1993 A
5188126 Fabian et al. Feb 1993 A
5190059 Fabian et al. Mar 1993 A
5197476 Nowacki et al. Mar 1993 A
5197965 Cherry et al. Mar 1993 A
5198768 Keren Mar 1993 A
5198877 Schulz Mar 1993 A
5202670 Oha Apr 1993 A
5211164 Allen May 1993 A
5211165 Dumoulin et al. May 1993 A
5211176 Ishiguro et al. May 1993 A
5212720 Landi et al. May 1993 A
5214615 Bauer May 1993 A
5219351 Teubner et al. Jun 1993 A
5222499 Allen et al. Jun 1993 A
5228442 Imran Jul 1993 A
5233990 Barnea Aug 1993 A
5237996 Waldman et al. Aug 1993 A
5249581 Horbal et al. Oct 1993 A
5251127 Raab Oct 1993 A
5251635 Dumoulin et al. Oct 1993 A
5253647 Takahashi et al. Oct 1993 A
5255680 Darrow et al. Oct 1993 A
5257636 White Nov 1993 A
5265610 Darrow et al. Nov 1993 A
5265611 Hoenig et al. Nov 1993 A
5269759 Hernandez et al. Dec 1993 A
5271400 Dumoulin et al. Dec 1993 A
5273025 Sakiyama et al. Dec 1993 A
5274551 Corby, Jr. Dec 1993 A
5279309 Taylor et al. Jan 1994 A
5291199 Overman et al. Mar 1994 A
5295483 Nowacki et al. Mar 1994 A
5297549 Beatty et al. Mar 1994 A
5299254 Dancer et al. Mar 1994 A
5299288 Glassman et al. Mar 1994 A
5305091 Gelbart et al. Apr 1994 A
5305203 Raab Apr 1994 A
5309913 Kormos et al. May 1994 A
5315630 Sturm et al. May 1994 A
5316024 Hirschi et al. May 1994 A
5318025 Dumoulin et al. Jun 1994 A
5320111 Livingston Jun 1994 A
5325728 Zimmerman et al. Jul 1994 A
5325873 Hirschi et al. Jul 1994 A
5329944 Fabian et al. Jul 1994 A
5333168 Fernandes et al. Jul 1994 A
5353795 Souza et al. Oct 1994 A
5353800 Pohndorf et al. Oct 1994 A
5353807 DeMarco Oct 1994 A
5368030 Zinreich et al. Nov 1994 A
5375596 Twiss et al. Dec 1994 A
5377678 Dumoulin et al. Jan 1995 A
5383454 Bucholz Jan 1995 A
5385146 Goldreyer Jan 1995 A
5385148 Lesh et al. Jan 1995 A
5386828 Owens et al. Feb 1995 A
5389101 Heilbrun et al. Feb 1995 A
5391199 Ben-Haim Feb 1995 A
5394457 Leibinger et al. Feb 1995 A
5397329 Allen Mar 1995 A
5399146 Nowacki et al. Mar 1995 A
5400384 Fernandes et al. Mar 1995 A
5402801 Taylor Apr 1995 A
5408409 Glassman et al. Apr 1995 A
5417210 Funda et al. May 1995 A
5419325 Dumoulin et al. May 1995 A
5423334 Jordan Jun 1995 A
5425367 Shapiro et al. Jun 1995 A
5425382 Golden et al. Jun 1995 A
5426683 O'Farrell, Jr., et al. Jun 1995 A
5426687 Goodall et al. Jun 1995 A
5427097 Depp Jun 1995 A
5429132 Guy et al. Jul 1995 A
5433198 Desai Jul 1995 A
RE35025 Anderton Aug 1995 E
5437277 Dumoulin et al. Aug 1995 A
5443066 Dumoulin et al. Aug 1995 A
5443489 Ben-Haim Aug 1995 A
5444756 Pai et al. Aug 1995 A
5445144 Wodicka et al. Aug 1995 A
5445150 Dumoulin et al. Aug 1995 A
5445166 Taylor Aug 1995 A
5446548 Gerig et al. Aug 1995 A
5446799 Tuy Aug 1995 A
5447154 Cinquin et al. Sep 1995 A
5448610 Yamamoto et al. Sep 1995 A
5453686 Anderson Sep 1995 A
5456718 Szymaitis Oct 1995 A
5458718 Venkitachalam Oct 1995 A
5464446 Dreessen et al. Nov 1995 A
5478341 Cook et al. Dec 1995 A
5478343 Ritter Dec 1995 A
5480422 Ben-Haim Jan 1996 A
5483961 Kelly et al. Jan 1996 A
5485849 Panescu et al. Jan 1996 A
5487391 Panescu Jan 1996 A
5487729 Avellanet et al. Jan 1996 A
5487757 Truckai et al. Jan 1996 A
5490196 Rudich et al. Feb 1996 A
5494034 Schlondorff et al. Feb 1996 A
5503416 Aoki et al. Apr 1996 A
5513637 Twiss et al. May 1996 A
5515160 Schulz et al. May 1996 A
5517990 Kalfas et al. May 1996 A
5531227 Schneider Jul 1996 A
5531520 Grimson et al. Jul 1996 A
5542938 Avellanet et al. Aug 1996 A
5543951 Moehrmann Aug 1996 A
5546940 Panescu et al. Aug 1996 A
5546949 Frazin et al. Aug 1996 A
5546951 Ben-Haim Aug 1996 A
5551429 Fitzpatrick et al. Sep 1996 A
5558091 Acker et al. Sep 1996 A
5568809 Ben-haim Oct 1996 A
5572999 Funda et al. Nov 1996 A
5573533 Strul Nov 1996 A
5575794 Walus et al. Nov 1996 A
5583909 Hanover Dec 1996 A
5588430 Bova et al. Dec 1996 A
5592939 Martinelli Jan 1997 A
5595193 Walus et al. Jan 1997 A
5596228 Anderton et al. Jan 1997 A
5600330 Blood Feb 1997 A
5603318 Heilbrun et al. Feb 1997 A
5617462 Spratt Apr 1997 A
5617857 Chader et al. Apr 1997 A
5619261 Anderton Apr 1997 A
5622169 Golden et al. Apr 1997 A
5622170 Schulz Apr 1997 A
5627873 Hanover et al. May 1997 A
5628315 Vilsmeier et al. May 1997 A
5630431 Taylor May 1997 A
5636644 Hart et al. Jun 1997 A
5638819 Manwaring et al. Jun 1997 A
5640170 Anderson Jun 1997 A
5642395 Anderton et al. Jun 1997 A
5643268 Vilsmeier et al. Jul 1997 A
5645065 Shapiro et al. Jul 1997 A
5647361 Damadian Jul 1997 A
5662111 Cosman Sep 1997 A
5664001 Tachibana et al. Sep 1997 A
5674296 Bryan et al. Oct 1997 A
5676673 Ferre et al. Oct 1997 A
5681260 Ueda et al. Oct 1997 A
5682886 Delp et al. Nov 1997 A
5690108 Chakeres Nov 1997 A
5694945 Ben-Haim Dec 1997 A
5695500 Taylor et al. Dec 1997 A
5695501 Carol et al. Dec 1997 A
5697377 Wittkampf Dec 1997 A
5702406 Vilsmeier et al. Dec 1997 A
5711299 Manwaring et al. Jan 1998 A
5713946 Ben-Haim Feb 1998 A
5715822 Watkins Feb 1998 A
5715836 Kliegis et al. Feb 1998 A
5718241 Ben-Haim et al. Feb 1998 A
5727552 Ryan Mar 1998 A
5727553 Saad Mar 1998 A
5729129 Acker Mar 1998 A
5730129 Darrow et al. Mar 1998 A
5730130 Fitzpatrick et al. Mar 1998 A
5732703 Kalfas et al. Mar 1998 A
5735278 Hoult et al. Apr 1998 A
5738096 Ben-Haim Apr 1998 A
5741214 Ouchi et al. Apr 1998 A
5742394 Hansen Apr 1998 A
5744953 Hansen Apr 1998 A
5748767 Raab May 1998 A
5749362 Funda et al. May 1998 A
5749835 Glantz May 1998 A
5752513 Acker et al. May 1998 A
5755725 Druais May 1998 A
RE35816 Schulz Jun 1998 E
5758667 Slettenmark Jun 1998 A
5762064 Polvani Jun 1998 A
5767699 Hansen al. Jun 1998 A
5769789 Wang et al. Jun 1998 A
5769861 Vilsmeier Jun 1998 A
5772594 Barrick Jun 1998 A
5775322 Silverstein et al. Jul 1998 A
5776064 Kalfas et al. Jul 1998 A
5782765 Jonkman Jul 1998 A
5787886 Kelly et al. Aug 1998 A
5792055 McKinnon Aug 1998 A
5795294 Luber et al. Aug 1998 A
5797849 Vesely et al. Aug 1998 A
5799055 Peshkin et al. Aug 1998 A
5799099 Wang et al. Aug 1998 A
5800352 Ferre et al. Sep 1998 A
5800535 Howard, III Sep 1998 A
5802719 O'Farrell, Jr., et al. Sep 1998 A
5803089 Ferre et al. Sep 1998 A
5807252 Hassfeld et al. Sep 1998 A
5810728 Kuhn Sep 1998 A
5810735 Halperin et al. Sep 1998 A
5823192 Kalend et al. Oct 1998 A
5823958 Truppe Oct 1998 A
5828725 Levinson Oct 1998 A
5829444 Ferre et al. Nov 1998 A
5831260 Hansen Nov 1998 A
5833608 Acker Nov 1998 A
5834759 Glossop Nov 1998 A
5836954 Heilbrun et al. Nov 1998 A
5840024 Taniguchi et al. Nov 1998 A
5840025 Ben-Haim Nov 1998 A
5843076 Webster, Jr. et al. Dec 1998 A
5848967 Cosman Dec 1998 A
5851183 Bucholz Dec 1998 A
5865846 Bryan et al. Feb 1999 A
5868674 Glowinski et al. Feb 1999 A
5868675 Henrion et al. Feb 1999 A
5871445 Bucholz Feb 1999 A
5871455 Ueno Feb 1999 A
5871487 Warner et al. Feb 1999 A
5873822 Ferre et al. Feb 1999 A
5884410 Prinz Mar 1999 A
5891034 Bucholz Apr 1999 A
5891157 Day et al. Apr 1999 A
5904691 Barnett et al. May 1999 A
5913820 Bladen et al. Jun 1999 A
5920395 Schulz Jul 1999 A
5921992 Costales et al. Jul 1999 A
5923727 Navab Jul 1999 A
5928248 Acker Jul 1999 A
5938603 Ponzi Aug 1999 A
5938694 Jaraczewski et al. Aug 1999 A
5947981 Cosman Sep 1999 A
5950629 Taylor et al. Sep 1999 A
5951475 Gueziec et al. Sep 1999 A
5954647 Bova et al. Sep 1999 A
5964796 Imran Oct 1999 A
5967980 Ferre et al. Oct 1999 A
5968047 Reed Oct 1999 A
5971997 Guthrie et al. Oct 1999 A
5976156 Taylor et al. Nov 1999 A
5980535 Barnett et al. Nov 1999 A
5983126 Wittkampf Nov 1999 A
5987349 Schulz Nov 1999 A
5987960 Messner et al. Nov 1999 A
5999837 Messner et al. Dec 1999 A
5999840 Grimson et al. Dec 1999 A
6001130 Bryan et al. Dec 1999 A
6006126 Cosman Dec 1999 A
6016439 Acker Jan 2000 A
6019725 Vesely et al. Feb 2000 A
6024695 Greenberg et al. Feb 2000 A
6050724 Schmitz et al. Apr 2000 A
6059718 Taniguchi et al. May 2000 A
6063022 Ben-Haim May 2000 A
6073043 Schneider Jun 2000 A
6104944 Martinelli Aug 2000 A
6118845 Simon et al. Sep 2000 A
6122538 Sliwa, Jr. et al. Sep 2000 A
6131396 Duerr et al. Oct 2000 A
6139183 Graumann Oct 2000 A
6149592 Yanof et al. Nov 2000 A
6156067 Bryan et al. Dec 2000 A
6161032 Acker Dec 2000 A
6167296 Shahidi Dec 2000 A
6172499 Ashe Jan 2001 B1
6175756 Ferre et al. Jan 2001 B1
6223067 Vilsmeier Apr 2001 B1
6233476 Strommer et al. May 2001 B1
6246231 Ashe Jun 2001 B1
6273896 Franck et al. Aug 2001 B1
6298262 Franck et al. Oct 2001 B1
6332089 Acker et al. Dec 2001 B1
6341231 Ferre et al. Jan 2002 B1
6351659 Vilsmeier Feb 2002 B1
6381485 Hunter et al. Apr 2002 B1
6424856 Vilsmeier et al. Jul 2002 B1
6428547 Vilsmeier et al. Aug 2002 B1
6434415 Foley et al. Aug 2002 B1
6437567 Schenck et al. Aug 2002 B1
6445943 Ferre et al. Sep 2002 B1
6470207 Simon et al. Oct 2002 B1
6474341 Hunter et al. Nov 2002 B1
6477228 Spahn Nov 2002 B2
6493573 Martinelli et al. Dec 2002 B1
6493575 Kesten et al. Dec 2002 B1
6498944 Ben-Haim et al. Dec 2002 B1
6499488 Hunter et al. Dec 2002 B1
6527443 Vilsmeier et al. Mar 2003 B1
6529758 Shahidi Mar 2003 B2
6551325 Neubauer et al. Apr 2003 B2
6567690 Giller et al. May 2003 B2
6584174 Schubert et al. Jun 2003 B2
6609022 Vilsmeier et al. Aug 2003 B2
6611700 Vilsmeier et al. Aug 2003 B1
6640128 Vilsmeier et al. Oct 2003 B2
6694162 Hartlep Feb 2004 B2
6695786 Wang et al. Feb 2004 B2
6701179 Martinelli et al. Mar 2004 B1
Foreign Referenced Citations (29)
Number Date Country
964149 Mar 1975 CA
3042343 Jun 1982 DE
3831278 Mar 1989 DE
4233978 Apr 1994 DE
10085137 Nov 2002 DE
0 319 844 Jan 1988 EP
0419729 Sep 1989 EP
0350996 Jan 1990 EP
0 651 968 Aug 1990 EP
0 581 704 Jul 1993 EP
0655138 Aug 1993 EP
0894473 Jan 1995 EP
2417970 Feb 1979 FR
2765738 Jun 1998 JP
WO 8809151 Dec 1988 WO
WO 8905123 Jun 1989 WO
WO 9103982 Apr 1991 WO
WO 9104711 Apr 1991 WO
WO 9107726 May 1991 WO
WO 9203090 Mar 1992 WO
WO 9206645 Apr 1992 WO
WO 9404938 Mar 1994 WO
WO 9423647 Oct 1994 WO
WO 9424933 Nov 1994 WO
WO 9611624 Apr 1996 WO
WO 9808554 Mar 1998 WO
WO 9838908 Sep 1998 WO
WO 9960939 Dec 1999 WO
WO 0130437 May 2001 WO
Related Publications (1)
Number Date Country
20030163038 A1 Aug 2003 US