Patent Application
20100177185
The present invention relates to a surgical microscope with integrated structured illumination and to reference display systems that may be used to project or display additional information, as e.g. provided from three-dimensional imaging or beamer devices, on the surface of an object viewed by an operating microscope e.g. during an operative procedure on a patient, e.g. by directly projecting optical information obtained e.g. from an MR or CT scan onto the patient or an object to be viewed using the microscope.
Several technologies, as e.g. computer tomographic imaging, stereotaxy, and microsurgery have rapidly evolved as important tools in clinical neurosurgery. Advances in imaging techniques (CT, PET, MRI . . . ) now provide three-dimensional information about anatomic and pathologic structures previously not realized. Stereotaxy, through use of a three-dimensional coordinate system and a mechanical frame, now allows the delivery of a probe to an intracranial target with great accuracy. The operating microscope provides the magnification and illumination to enable surgeons to work with significantly greater precision and safety.
At present, a neurosurgeon's ability to perform intracranial procedures for tumor, vascular disease or functional disorder is dependent upon his mental integration of the visualized operative field with his knowledge of neuroanatomy and the available radiologic studies such as CT or MR scans.
Conventional CT scans are oriented transversely to the body axis and as the operative approach is rarely along the axis of conventional scanning, the ability to reconstruct a CT scan to match the surgical perspective is highly appealing.
CT scanning and reconstruction have become an integral and important part of modern medicine; the procedure used involves primarily image processing software and computer graphics. Adaptation of stereotactic technique to CT technology has been approached in a number of ways. One technique utilizes an adapted metal frame fixed to the patient's head at the time of scanning. Stereotactic coordinates, relating the target position of CT-demonstrated pathology to the stereotactic instrument, are generated directly from the scans and the patient is then transported to the operating room. Other developed techniques are adequate but often more cumbersome. All of these enable stereotactic procedures generally characterized by “blind” insertion of needle-like instruments through small openings utilizing previously obtained CT-determined landmarks. Earlier developments have not generally been amenable to “open” procedures such as craniotomy for tumor or vascular disease and, as previously noted, do not allow access to CT data after selection of a target. The CT information utilized is limited to the coordinates of a point. All instruments select a target and set the instrument accordingly; the present invention, operating in reverse, allows free positioning of the microscope with subsequent stereotactic positioning the additional data, as e.g. CT data or MR data, directly on an object or patient.
The operating microscope has been incorporated into CT stereotactic work described in two formal articles: “A Stereotactic Approach to Deep-Seated Central Nervous System Neoplasms Using the Carbon Dioxide Laser” (Surg-Neurol, 15: 331-334, 1981, Kelly et al.); and “A Microstereotactic Approach to Deep-Seated Arteriovenous Malformations,” (Surg-Neurol, 17: 260-262, 1982, Kelly et al.). The Kelly et al. development has also employed surgical laser instrumentation and shows the feasibility of achieving a synthesis of technologies and the prospects of refining neurosurgical operative techniques. Their technique of linking the operating microscope and the stereotactic reference system requires utilization of a large stereotactic frame and does not employ a projection system.
Although the present system is discussed herein mostly in the context of neurosurgery, e.g. craniotomy, it is useful in other operating contexts.
U.S. Pat. No. 6,466,815 B1 discloses a navigation apparatus comprising a navigation-related information generating section and a display section. The navigation-related information generating section measures the position and orientation of an object and a target in a three-dimensional space and generates navigation-related information to be used for navigating the object toward the target. The display section displays the navigation-related information generated by the navigation-related information generating section in any of different modes depending on the relationship of the position and orientation of the object and that of the target. A surgical operation image acquisition/display apparatus comprises an observation section, an image display section and a specifying section. The observation section includes a plurality of observation sections whose position and orientation is modifiable. The image display section is adapted to alternatively display any of the images obtained by the observation sections or synthetically combine and display the combined images. The specifying section specifies the image to be displayed to the image display section according to the position and orientation of the observation section.
U.S. Pat. No. 6,577,080 B2 discloses methods and systems for signal processing for illumination control signals, which may include a decoder for decoding a combined signal and a connection for delivering a portion of the combined signal to an illumination source, which source is capable of generating an illumination condition from the portion of the combined signal. The system may also include an encoder for encoding the combined signal from an illumination control signal and a second signal. The illumination source may be an LED system that is controlled by a microprocessor to vary at least one of the color and intensity of the illumination produced by the illumination source in response to the illumination control signal.
U.S. Pat. No. 5,769,861 discloses a method of localizing an instrument relative to three-dimensional corporeal data, particularly for stereotaxis, comprising the following steps: connecting internal markers fixedly to the body to define an intracorporeal, spatial reference system, implementing an analytical scan of the body including the internal markers to determine the positions of the three-dimensional corporeal data obtained from analytical scan in the intracorporeal reference system defined by the internal markers, implementing a referencing step, whereby position and orientation of the intracorporeal reference system defined by the internal markers relative to an extracorporeal reference system defined by external markers is determined and said external markers is located outside of the body in a fixed spatial relationship, determining the position and orientation of an instrument in the extracorporeal reference system with the aid of the external markers and a instrument markers attached to the instrument, and computing from said position and orientation in the extracorporeal reference system the position and orientation of the instrument in the intracorporeal reference system via the relationship between the intracorporeal and the extracorporeal reference systems known from the referencing step, as well as a markers device and a device for referencing.
U.S. Pat. No. 6,351,659 B1 and U.S. Pat. No. 6,859,660 B2 disclose a Neuro-navigation system comprising a reflector referencing system including passive reflectors and a marker system with markers or landmarks wherein the reflectors as well as the markers as regards their shape, size and material selection as well as their arrangement or attachment on the parts of the body to be operatively treated and on the surgical instruments are configured so that mapping their locations is substantially facilitated or is able to take place more accurately positioned by a computer/camera unit having a graphic display terminal as well as the operative treatment with the aid of this unit. Optionally a surgical microscope, an ultrasonic diagnostic system as well as a calibration procedure may be integrated in the Neuro-navigation system in accordance with the invention.
U.S. Pat. No. 6,236,875 B1 discloses a system for use during a medical or surgical procedure on a body. The system generates an image representing the position one or more body elements during the procedure using scans generated by a scanner prior or during the procedure. The image data set has reference points for each of the body elements, the reference points of a particular body element having a fixed spatial relation to the particular body element. The system includes an apparatus for identifying, during the procedure, the relative position of each of the reference points of each of the body elements to be displayed. The system also includes a processor for modifying the image data set according to the identified relative position of each of the reference points during the procedure, as identified by the identifying apparatus, said processor generating a displaced image data set representing the position of the body elements during the procedure. The system also includes a display utilizing the displaced image data set generated by the processor, illustrating the relative position of the body elements during the procedure. Methods relating to the system are also disclosed. Also disclosed are devices for use with a surgical navigation system having a sensor array which is in communication with the device to identify its position. The device may be a reference frame for attachment of a body part of the patient, such as a cranial reference arc frame for attachment to the head on a spine reference arc frame for attachment to the spine. The device may also be a localization frame for positioning an instrument relative to a body part, such as a localization biopsy guide frame for positioning a biopsy needle, a localization drill guide assembly for positioning a drill bit, a localization drill yoke assembly for positioning a drill, or a ventriculostomy probe for positioning a catheter.
U.S. Pat. No. 4,722,056 discloses a reference display system that receives information from an imaging system (e.g., a CT scanner or the like), that extracts or derives three-dimensional anatomical and/or pathological information about a part of a body (e.g., the brain or other organ) of a patient. The information is digitized in the imaging system and is introduced to a computer that is programmed to reformat the digitized information to provide as output electric signal representative of the digitized information. An optical display system (e.g., a cathode ray tube, CRT, and related circuitry) is connected to receive the output of the computer and is operable to present the reformatted information at a determined plane during an operative procedure. An operating microscope is freely located in the operative location relative to the patient during the operative procedure, the focal plane of the microscope establishing the determined plane. A way is provided to establish the spatial relationship among the imaging system, the patient, and the focal plane of the microscope; and a mechanism is provided to project the reformatted imaging system information into the optics and onto the focal plane of the operating microscope during the operative procedure, the reformatted image being displayed as an overlay upon the optical image of the body part on which the operation is being performed.
The present invention integrates external or additional information, obtained e.g. from CT or MR, and computer technology and preferably stereotactic principles and the operating microscope, to develop a computer-based optical system for use during microneurosurgical procedures. This technique has the capability to incorporate the available computer technology with the operating room environment and many neurosurgical procedures.
An objective of the present invention is to provide a system that will project and display reconstructed CT images on at least part of the surface of an object preferably in the field of view of the operating microscope. The neurosurgeon will then see, for example, the outline of a tumor (reconstructed by a computer) superposed on the operative field.
This provides advantages, since e.g. no dependence on the surgeon's mental reorientation of CT scanner information exists. The information can be displayed such that it will not interfere with the neurosurgeon's procedure or require the reading of x-rays off a light screen some distance away. A computer-based anatomical atlas can be developed that will superpose on the operative field important, but otherwise unseen, structures such as normal neuronal pathways and nuclei and major vascular structures. The neurosurgeon can use the projected image(s) as a map accurately guiding operative procedures with greater precision than presently possible.
Another objective of the present invention is to provide a system that will superpose appropriately reformatted, three-dimensional imaging information on the object or patient by projecting the generated optical information on the object which is visualized through the operating microscope.
A still further objective is to provide the capability to present information of structure lying directly below the focal plane of the microscope by providing an optical information directly on the object, so the surgeon can find the location on the surface directly above the region of interest.
Another objective is to present to the surgeon accurate information on the boundary between different tissues e.g. by displaying on the object the boundary and/or by displaying different regions in different colours so the surgeon can locate such boundaries accurately; for example, the surgeon may be debulking a tumor and wish to know where the edge of the tumor is relative to normal brain tissue.
An additional objective is to overlay or project on the object being in the visual field of the microscope, generic information from an atlas of information about functional purposes of different regions of the brain.
The foregoing objectives are attained generally in a reference display system and method involving an operating microscope and projector or beamer for projecting and/or displaying information e.g. from a three-dimensional imaging device or an atlas during an operative procedure on a patient, that includes: an imaging system (e.g., CT, MRI, PET, or ultrasound scanners) that contains three-dimensional anatomical and/or pathological information about a part of the body (e.g., the brain or other organs) of the patient; means to digitize the information; a computer connected to receive the digitized information (e.g., stored on tape, direct transmission, or other ways), the computer being programmed to reformat the stored information to present the same at a determinable plane being on the object or patient viewed by the microscope; an operating microscope positioned in the operative location relative to the patient, the focal plane of the microscope preferably coinciding with said determinable plane; means for determining the spatial relationships of the information obtained from the imaging system, the surface of the object or patient, and the focal plane of the microscope or the field of view of the microscope or the microscope itself or projecting device or beamer with respect to one another; and means to project the reformatted imaging system information onto the object or patient to be viewed by the operating microscope.
The invention allows thus a flexible illumination or projection of information on the object in the field of view of a surgical microscope, which leads to information enrichment during surgical procedures.
By replacing simple light sources previously used in connection with a microscope or adding an additional “intelligent” light source or beamer being for example a LCD beamer, a xenon lamp or a laser projector, different light setups and displays of additional information directly on the surface of the object being observed through the microscope can be provided.
The surgical microscope comprises according to a first embodiment a light source, such as a laser projector or LCD-beamer, which generates light within the microscope or which light can be transmitted to the microscope using for example an optical fiber. This light is projected onto the surface of the object to be viewed by the microscope using preferable the objective lense of the microscope to project, emit or focus the light generated by this light source onto the object. Furthermore, a control unit is provided to generate signals or images to be projected onto the object to control the light source of the surgical microscope, so that the images projected onto the object are preferably in a predetermined or corresponding spatial relationship to the object preferably matching the structures or surface of the viewed object. Thus, a structured illumination can be provided.
The light emitted by the light source can be guided to the object of the microscope using preferably a prism and/or a semi-transparent mirror being present in the microscope, which prism or semi-transparent mirror can also be used for deflecting or guiding the image of the viewed object to the ocular of the microscope to be seen by the observer. The surgical microscope consists according to a second embodiment of an optics carrier, a stand, a light source that can be built by an external laser projector, and a control unit.
Optional, one or more cameras that observes the field of view of the microscope and a surgical navigation system can be added for either embodiment to control the projector itself or to control the control unit.
For the advanced features in combination with the navigation system as set forth below, the microscope and preferably the optional camera(s) and the projector have to be calibrated to allow the spatially correct display of information.
The following setups could be imagined and of course others could be possible:
Plain light: similar to the conventional microscope light, monochrome light, adjustable in color and intensity, can be emitted. The color adjustment could for example be used to accentuate specific tissue.
Simple shaped light: Like an advanced aperture, the light cannot only be restricted having a circular shape, but for every arbitrary shape at the outline the projector can handle. This can be useful to blend out useless information, e.g. in neurosurgery the light can be adjusted to just enlighten the craniotomy.
Complex shaped light: Different parts of the field of view can be enlightened with different light parameters, e.g. different colors and/or different brightness.
Information enrichment: The light source, e.g. xenon lamp, bamer (having e.g. a built-in LCD-device) or laser projector, can be used to project information (e.g. text, images, contours, . . . ) directly onto the object (e.g. a human brain) that is watched through the microscope. This is similar to the built in displays also known as Head-up Displays (HUD) of modern surgical microscopes but has some advantages.
A conventional microscope HUD is inconvenient to handle for some surgeons because of the different distances of the HUD and the real life object to the surgeon's eye. Some surgeons can compensate for that but some cannot. With the integrated laser projector this problem does not occur because the information is displayed directly on the real life object.
A conventional microscope HUD has very limited contrast because the built in light source of the microscope outshines the information displayed on the HUD. Because of the combination of the light source and the information display, this invention allows to display the information with a high contrast without darkening the whole field of view by dimming the light source.
By combining this setup with an optionally color and/or spatially calibrated video camera that records the field of view of the microscope, even more sophisticated applications are possible, e.g. the video information can be used to automatically adjust the light parameters (e.g. light color depending on the tissue color). When two cameras are used (or one camera at different positions) the three dimensional structure of the field of view can be determined. This information can be incorporated in the algorithms controlling the projector (e.g. projecting information related to the spatial position of the part of the object part of the information is projected on).
The combination with the video camera can also be used to display images that have been recorded beforehand, e.g. the image of the object in the field of view of the microscope is recorded at special lighting conditions (e.g. tumor markers in fluorescent light) and this information is redisplayed later at different lighting conditions (e.g. “normal” light) to combine multiple lighting conditions in one.
When the information display of the projector is separated for the left and right ocular (and therefore for the left and right eye), also a stereoscopic projection is possible. One possible implementation would be two shutters integrated in the oculars where at each point in time only one shutter is open and at the same point in time the projector generates the corresponding image for the open ocular.
This device can be combined with a surgical navigation system to display anatomical data on the real world object at the correct position. The information available by the navigation system can be used to adapt the projector. For example the information of the shape of the head of the patient and the information of the position of this head in relation to the (calibrated) microscope and therefore to the (calibrated) projector can be used to adapt the projected shape.
The laser projector could also be used as an additional component to be added to a conventional microscope in addition to the conventional light source. This would not provide the full power as the replacement of the standard light source regarding light shaping etc., but the information overlay over the real world object is still advantageous compared to a conventional HUD e.g. regarding the accommodation problem.
The laser can be tuned to frequencies that meet the resonance of diagnostically or treatment related applied flourescent markers, so that these markers selectively light up and/or are activated.
The laser can be tuned to an energy where it can be used for heating up the tissue (coagulation, ablation . . . ).
The invention is hereafter described with reference to the accompanying drawings in which:
To simplify this explanation, the three-dimensional imaging device 1 will be considered to be images from a CT scanner. The interacting elements in the system serve to project and optionally display three-dimensional anatomical and/or pathological information derived from the brain, or other organ or body part, by the scanner 1 on a part of the surface of the patient as visualized through the microscope 2 to present a composite view 12 to the surgeon viewing the patient 18 with the image 9 projected thereon.
The scanner 1 provides image information that is stored, e.g., on x-ray film or digital tape or by a first computer 4. If not in digital form, the information is digitized and the digitized information is connected as input at 3 to the first computer 4 that acts as a treatment planning computer that provides an output 5 that is transferred to a second computer 6. The computers 4 and 6 can be the same or different computers. The scanner 1 can provide real time data to the computer 4, but, typically, the information is stored on magnetic tape and, later, it is introduced to the computer 4 for processing. The computer 6 takes the information from the computer 4 and information from reference system 11, such as e.g. VectorVision® provided by BrainLAB®, that identifies the spatial position of the patient 18 having e.g. markers 18a.
The computer 6 is programmed to reformat the digitized information taking into account the relative position between an optical display system or beamer 7 and the patient 18 and to provide an output signal at 8 representing the reformatted information. The information at 8 eventually is fed to the optical display system 7 (e.g. laser beamer (monochrome or colour) or any other optical projecting means) which forms an image that is displayed on the surface of the patient 18.
Assuring that the image of the brain region generated by the computer 6 and displayed by laser beamer 7 registers with the surface of the patient 18 is an important aspect of the present invention, as discussed later.
Whereas the signals at 3, 5, and 8 are electrical signals (i.e., either analog or binary), the signal output at 9 of the laser beamer 7 is an optical signal which conveys the reformatted information as an image directly to the patient 18. The neurosurgeon can see both images 9 and 18 as combined image 12 by only viewing the patient 18 since they are superposed upon one another.
The computer-generated image projected by beamer 7 is displayed on the patient 18 and provides one combined image 12 that the surgeon sees. The image should be bright enough to provide an acceptable superposed image on the patient 18 or object in the illuminated operative field. The laser beamer 7 also has size constraints, since it is preferably mounted on the microscope 2, as seen in
The beamer 7 can e.g. be attached to a standard operating room microscope 2 so that the beamer 7 is able to generate an optical image 9 on an object 18 in the field of view of the microscope 2.
The micrsoscope 2 has an ocular 15, as shown in
Reference numeral 16 designates a magnification changer containing a prism or semi-transparent mirror 13 receiving light to illuminate the object 18 from a light source 19, as for example a xenon lamp, transmitted through an optical fibre 14 and deflected by the prism 13 onto the object 18. Prism or semi-transparent mirror 13 also serves to deflect the combined image 12 to be sent to ocular 15 viewed by the surgeon.
It will be appreciated, on the basis of the foregoing explanation, that the computer-generated image of the brain region of the patient can be precisely focussed and projected onto the surface of head or brain 18, because the beamer 7 is rigidly attached to the microscope 2. It remains, therefore, to bring the optical image of that same brain region to the projection plane of beamer 7 being the surface of the object 18 to effect registration of the images with object 18. This requires precise knowledge as to the spatial location of the beamer 7 or the microscope 2 carrying beamer 7 and of the patient or object 18 relative to the beamer 7 or microscope 2, as explained below.
A scheme for registering the projection of a computer-generated image of an object with the surface of the same part of the object 18, e.g. brain (or other body part) and viewed in the microscope 2 is described below. The aim is to present to the surgeon a computer-generated or reformatted image (from a CT scan or other imaging system) projected in proper orientation and scale upon the object 18, e.g. the brain. Thus, establishing the spatial relationship among the object or patient 18, and the focal or projection plane of the beamer 7, is important.
Exactly projecting the images is accomplished by spatially relating both the imaging data and the beamer 7 (or the operating microscope 2 with attached beamer 7) to the same points (hereinafter called markers) 18a on the patient 18. The general registration procedure can involve CT scanning the patient 18 with at least three markers 18a attached to the head of the patient 18 (the patient may be supine or seated). The markers 18a are composed of a material (e.g., tungsten), that is physically detectable by the imaging system (i.e., the CT scan at this part of the explanation) during scanning as well as visually to the microscope 2 or an IR-camera 11 to achieve registration of the computer-generated projected image from the CT scan with the patient 18 being preferably at the focal plane of the microscope 2.
A preferred way to establish the spatial relationship between the beamer 7 (or microscope 2) and the markers 18a, as well as to track any subsequent movement of the microscope 2 or beamer 7, includes optical tracking of the markers 18a attached to the object or patient 18 and markers 7a attached to the beamer 7 and/or microscope 2. Determining the spatial relationship of two objects 7 and 18 each bearing markers 7a and 18a is well known in the art.
The reference system 11, such as the mentioned VectorVision® system, can determine the position of the microscope 2 or beamer 7 and its focal plane with respect to the patient's head 18 and CT scans.
The reconstructed CT scan, as above indicated, must be projected by beamer 7 as a two-dimensional CRT image. This involves converting the reconstructed slice from a matrix of image data in three coordinates to one of two coordinates (x, y). A microscope or beamer coordinate system could represent the projection or focal plane as x and y, normal to the optical axis with the origin at the focal point. This technique requires a transformation of coordinates because the beamer or microscope coordinate system will be constantly changing with respect to the location of the object 18 and corresponding CT scan data of the object 18 as the surgeon moves the microscope 2. Regardless of the reference frame used for reconstructing the slice, in order to project and thus display the proper image, the slice must be transformed into beamer or microscope coordinates. The advantage of transforming all the data and performing the calculations in the microscope (with attached beamer) coordinate system is that if the surgeon moves the microscope 2 only slightly or along the optical axis, the reconstruction calculations can be greatly reduced and allow the surgeon to quickly call up new slices for projection.
Unlike the embodiment shown in
Thus, an external or separate light source, such as laser beamer 7 shown in the embodiment of
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/EP07/05334 | 6/18/2007 | WO | 00 | 12/15/2009 |
