Patent Application
20050117118
The present invention relates to a method and device for providing computer-generated aids for ophthalmic surgery, and more particularly, for visualization of ophthalmic structures.
During an ophthalmic surgical procedure using a standard operating microscope, a surgeon may need to make various measurements, such as incision length, the size of an anatomic structure, or distance between anatomic structures. Such measurements are either done by superimposing a reticle in the eyepiece over the area under consideration, or by using a ruler and making a direct measurement in the field of operation. In both cases, the operator often must estimate if the edge of the structure measured falls between the marks on the ruler or on the reticle.
Additionally, during the ophthalmic surgical procedure, the surgeon often marks areas to be excised or sutured with an inked marker. During the operation, the ink tends to be washed away because of frequent irrigations necessary to keep the surface of the eye moist, making such excisions or sutures difficult.
Furthermore, either when operating alone or with an assistant, a surgeon may need advice pertaining to the surgical procedure, such as when performing a new or complicated operation or when facing an unexpected complication. It is important that such advice be provided in a timely and reliable manner. Advisors in remote locations may not be able to observe the surgical subject, making it difficult to provide advice. Additionally, the level of expertise available may not be able to provide proper guidance.
In an embodiment of the present invention, a method of providing visual documentation for an ophthalmic surgical procedure is provided. The method includes providing at least one video camera, at least one monitor and a processor. An image of an eye is captured using the video camera. A template having a graphical content that is useful to the surgeon performing a surgical procedure is displayed contemporaneously with the image of the eye on the monitor.
In some embodiments of the invention, the captured image is a stereoscopic view pair and the image of the eye displayed on the monitor is a stereoscopic image. In other embodiments, the image is a single view image.
In a further embodiment of the invention, the image of the eye is captured such that reflections from structures in the eye are reduced. In one embodiment, the eye is illuminated from a given angle and the video camera that captures the image is positioned and aimed to minimize the reflected light captured by the camera. In other embodiments, illuminating light is sequenced from a series of angles and video cameras capture a sequence of images. The sequence of images is then processed to form a composite image with reduced reflections.
In other embodiments of the invention, a system that can respond to a wide dynamic range of illumination is provided to reduce the saturation of highlights and thereby to reduce the effects of glare. In some embodiments of the invention, the wide dynamic range is achieved using a video camera with adjustable sensitivity and using corresponding scaling techniques. In other embodiments, the wide dynamic range is achieved by illuminating the first eye with a variable brightness light source. The brightness of the light source may be adjusted by determining the illumination level at specified points in the captured image and then adjusting the illumination level. In some embodiments of the invention, this adjustment may be made with a feedback loop.
In various embodiments of the invention, the captured image may be processed to change the resolution, contrast, brightness, magnification and color of the image. The image may also be processed to enhance an edge in the image. In some embodiments of the invention, a portion of the image of the eye may be magnified at higher magnification level than another portion of the eye. The portion of the eye with the higher magnification level is shown in greater detail, while the rest of the image provides awareness to a surgeon of the situation outside of the highly magnified portion. In other embodiments of the invention, the eye may be illuminated with laser light. An opaque screen is provided to shield operating room personnel from exposure to the light and the surgeon may view the image of the eye on the display monitor, rather than directly. Thus, danger to sensitive tissues of operating room personnel may be reduced.
In other embodiments of the invention, the eye may be selectively illuminated with specified wavelengths or ranges of wavelengths of light. Further, the video camera that captures the image may have a variable response to light according to the wavelength. Thus, the image displayed to the surgeon on the monitor may emphasize certain wavelengths of light over others to highlight particular structures of the eye.
In other embodiments of the invention, a method is provided to measure distances on the eye. Two points in the image of the eye are identified: in some embodiments the points are identified on the display monitor using a pointing device. The distance between the points is then displayed on the monitor. In another embodiment of the invention, the template shows the major axes of astigmatism of the cornea and the optical center of the cornea. As a procedure progresses, these measurements may be updated and redisplayed for the surgeon.
In some embodiments of the invention, a template is displayed on the image of the eye on the monitor that show the position and extent of an incision to guide a surgeon. The incision may be for one of cataract removal and corneal repair. In other embodiments of the invention, the templates shows a suturing pattern. In other embodiments of the invention, the template includes an indication of the position and orientation for a surgical instrument during a procedure.
In other embodiments of the invention, a template may provide guidance for placing an implant, including a transplant, in an eye. The template may identify the placement and shape of the implant, so that the surgeon may make an appropriately sized incision at the site. Further, once a site has been prepared for an implant or a transplant, the surgeon may identify with an input device the size and shape of the prepared site. A template may then be generated by the processor to match the outline of the prepared site. This template may be superimposed on the image of the eye at the site where the transplant tissue is to be harvested, as a guide to the surgeon.
In other related embodiments of the invention, a template may include surgical advice pertinent to the surgical subject. Such advice may be interpretive. Surgical advice may include, for example, providing pattern recognition. The template may include a selectable suturing pattern into the template. Displaying the template overlaid contemporaneously with the image of the eye may include displaying a plurality of display windows on one or more displays. Audio pertinent to the surgical subject may be provided. In other embodiments of the invention, data may be received and displayed on the display monitor. The data may include data characterizing the status of a patient, or the status of equipment, or both. The data may be updated in real-time and may be displayed on a display surface that shows the image of the eye.
The foregoing features of the invention will be more readily understood by reference to the following detailed description, taken with reference to the accompanying drawings, in which:
In various embodiments of the invention, a method is provided for visualization of the structures of the eye. Video processing and digital image processing techniques enhance the image improving an ophthalmologist's ability to see the structures clearly. In specific embodiments of the invention, a computer is employed to process the captured eye images and to overlay information that is useful for the ophthalmologist for performing therapeutic procedures on the eye.
Stereoscopic video camera 109 may include, without limitation, a beam splitter 101 that splits an optical image of the surgical subject 110 into left and right images representing the stereoscopic view pair of subject 110. The left and right images are magnified by magnification-varying optical units 102L and 102R, and focused by lenses 103L and 103R onto imaging devices 104L and 104R, respectively. Imaging devices 104L and 104R, which may include, for example, a Charge-Coupled Device (CCD), convert the left and right optical images into electrical signals 105L and 105R, respectively. Electrical signal 105L is provided to image processor 106L which outputs a video signal 107L representing the left image of the subject, while electrical signal 105R is provided to image processor 106R which outputs a video signal 107R representing the right image of the subject 110. Note that in various embodiments, the stereoscopic camera may include, instead of a beam splitter, two separate cameras that are used to capture the image from two different angles.
The video signals 107L and 107R from stereoscopic camera are fed into a processor 108, which, in part, may perform image signal processing and/or format the signals 107L and 107R so as to drive the one or more displays 112. Processor 108 may include, without limitation, one or more microprocessors, programmable logic arrays, and/or other logic circuits, as known in the art.
Numerous methodologies known in the art may be used in displaying the three dimensional image. For example, and without limitation, a Liquid Crystal Display (LCD) may be placed over a display screen. The LCD produces fast alternating polarization that is synchronized with alternating presentation of the left and right images on the display screen. Also known are systems in which the left and right images are displayed simultaneously, with each image being polarized in a different configuration. For example, the left and right images may be displayed side-by-side.
Typically, a viewer of the display(s) 112, who may be a surgeon, an assistant surgeon, a nurse, an anesthesiologist and/or any other observer, wears special spectacles 113 so that the left eye receives the left image and the right eye receives the right image. The spectacles may be one of many types as known in the art, such as, but not limited to, special polarizing lenses or alternating occlusion lenses. Upon receiving the two images (i.e. the stereoscopic view pair), the viewer's brain triangulates the left and right images, as seen with the left and right eyes, respectively, such that the viewer perceives a three-dimensional image.
In other embodiments of the invention, a non-stereoscopic (i.e., monocular) view may be captured by a video camera and presented for viewing to a surgeon. A system similar to the system shown in
Each camera 202-204 may be removably mounted to a boom 211. Boom 211 may include, without limitation, a stand that is adjustable in height, and/or one or more arms that may move in an arbitrary direction. Thus, each camera's position can be properly adjusted so as to provide a desired view of a subject 210. In other embodiments of the invention, the camera(s) may be attached to a headpiece worn by a patient, so as to be focused on, for example, an eye upon which surgery is to be conducted. Thus, the camera(s) will remain focused on the eye regardless of any movement by the patient.
The images from each camera may be selectably displayed on one or more displays 205-207. Each display(s) 205-207 may have a high resolution of, for example, 1280 pixels×1024 pixels or greater. Display(s) 205-207 may be, without limitation, a monitor or a flat-panel display. Multiple images may be displayed on a single display, using, for example, a multi-window graphical operating system.
Through the use of video displays 205-207, the system 200 advantageously provides a larger field of view compared to a traditional operating microscope. With the traditional operating microscope, the size of the viewable field is inversely proportional to the level of magnification. Thus at higher magnifications, the size of the field or view is small. With video system 200, magnification is augmented by the size of the video display(s) 205-207. The optical magnification can be arranged to allow a larger visual field, and then extra magnification of the objects in the field is supplied by a larger display area. The larger display field allows the surgeon to notice potential problems at the periphery of the operative field, an area that may not be seen with a conventional microscope using high magnification. In some embodiments of the invention, as shown in
System 200 may include an operator interface 209. Operator interface 209 provides the surgeon and/or other observer (referred to hereinafter as the “surgeon”) the capability to control various aspects of system 200. For example and without limitation, the surgeon may control, via the operator interface 209, zoom functionality, focusing of cameras 202-204, and/or which camera image or other graphical content is displayed on a particular display. Graphical content on each display may be controlled by software driven menus or taskbars using point and click methodology or other similar means known in the art. Operator interface 209 may include, without limitation, a keyboard, a trackball, a joystick, and/or a mouse. Operator interface may also include a remote control, a foot control that frees the surgeon's hands, and/or one or more memory devices, such as a semiconductor, magnetic, optical or other memory device.
The system 200 includes at least one illumination source 212. The illumination source 212 may be, without limitation, an incandescent bulb (e.g., a xenon arc bulb) that is typically used with traditional operating microscopes. However, incandescent bulbs are subject to burn-out and may need replacement during surgery. Instead of an incandescent bulb, a light-emitting diode (LED) may advantageously be utilized, in accordance with one embodiment of the invention. LEDs typically have a higher reliability than incandescent bulbs, and may function for thousands of hours before failure. Additionally, the reliability of the system can further be improved if a second illumination source is provided, which may be attached to an auxiliary camera. In certain surgeries, an eye may be illuminated by a laser, such as for laser vision correction. In such surgeries, an opaque barrier may be placed between the laser light and operating room personnel, including the surgeon. The surgeon may view the image of the eye on the display monitor, rather than viewing the eye directly. Thus, dangerous reflections of laser light from the eye or other surfaces to sensitive tissues of operating room personnel may be reduced.
The illumination source 212 provides enough light to produce a stereoscopic video image. This single image may then be provided and viewed simultaneously on a plurality of displays. Thus, compared to traditional microscopic systems, which typically require an optical beam splitter for each observer, with each beam splitter requiring additional illumination, less illumination is required. This is particularly important in eye surgery. For example, the retina of a patient can be damaged by high light intensity. Note that in various embodiments, the video image generated by the system 200 can be electronically brightened.
Since the illumination used in the system 200 for multiple observers is typically less than that used in traditional microscopic, as described above, and below the level of illumination that can cause damage to the eye, the illumination in the system 200 may be advantageously increased. For example, illumination may be increased (yet remain within safe limits), such that a polarization filter can be placed over the camera lens. The polarization filter can eliminate annoying reflections that can create glare and obscure key anatomic features during surgery.
In another embodiment of the invention, other methods of reducing reflections in the displayed image of an eye are provided.
In a further embodiment of the display, a method is provided for reducing the effects of glare in an image of an eye. The image is captured with a system with a wide dynamic range so that the saturation of highlights in portions of the image may be reduced. The wide dynamic range may be achieved, in some embodiments, by using a video camera with adjustable sensitivity, taking a sequence of images of the eye, and using scaling techniques for the various portions of the eye to form a composite image of the eye, thereby reducing the saturation of highlights. In some embodiments of the invention, the wide dynamic range is achieved by illuminating the eye with at least one variable brightness light source and taking a sequence of images of the eye under varying brightness levels. The sequence of the captured images is then processed to form a composite image, reducing the saturation of highlights. The brightness of light for a particular image in the sequence of images may be determined using a feedback loop.
The illumination source 212 may also be a slit lamp or an ophthalmoscope, with the microscope typically used in such systems replaced by a stereoscopic video camera.
The bright illumination from a light source, such as slit lamp or ophthalmoscope, can be annoying to a patient. With the three dimensional video system of the present invention, the video signal representing the stereoscopic view pair may advantageously be stored in memory and analyzed at a later time, so as to minimize the amount of time the eye is illuminated.
In embodiments of the invention, an eye may be illuminated at a selected wavelength or a range of wavelengths. Particular structures in the eye absorb or reflect light more strongly at certain wavelengths. For example, blood vessels just below the surface of the skin, such as the vessels in an eyelid, can be made visible using blue/green light since these wavelengths are strongly absorbed by hemoglobin. Other wavelengths (often in near ultraviolet or near infrared) can make tissue or certain in vivo dyes fluoresce. The illuminating light may be restricted to a wavelength or a range of wavelengths by, for example, applying filters to the illuminating source. Further, the image presented to the display 112 may be filtered, either at the video camera, or other sensor, or electronically. Such filtering can restrict the image viewed to certain wavelengths of light.
Referring back to
Processor 8 may perform image enhancement on the stereoscopic view pair prior to providing the stereoscopic view pair to the display(s) 112. Image enhancement may include, without limitation, contrast enhancement, enhancement of edges, zoom capability, electronic brightness control and/or providing special coloring to bring out features such that are difficult to detect with conventional microscopy, such as cataract fibers during cataract removal. Image stabilization may be provided, so as to present a stable image regardless of whether the subject is vibrating or otherwise moving.
In various embodiments of the invention, processor 108 may provide a template that has a graphical content pertinent to the subject of the stereoscopic view pair. The template may be displayed such that the template is overlaid contemporaneously with the stereoscopic view pair. For example, the template may be superposed on, and/or juxtaposed next to, the displayed stereoscopic view pair. All or portions of the template may be displayed so as to be perceived by the viewer in three dimensions. Template may be displayed on display(s) 112 in one or more window(s) in combination with, or separate from, the stereoscopic view pair. For example, and without limitation, the template may be displayed using a split screen and/or picture in picture approach.
A sample template 500 that incorporates the stereoscopic view pair 501 is shown in
In accordance with one embodiment of the invention, the viewer determines the measurement to be made using at least one cursor 504 and 505 that is overlaid on the displayed stereoscopic view 501. The cursor(s) 504 and 505 may be positioned, selected, and/or otherwise controlled using, for example, the operator interface 209 (see
The template 500 may provide changes in elevation pertaining to the displayed stereoscopic view pair. Processor 8 may include, for example, a stereogrammetry program, for measuring the elevation changes. This feature may be used, for example and without limitation, at the close of many ophthalmic operations wherein, after all the sutures have been tightened, sterile fluid is injected into the eye to restore normal intra ocular pressure. If too much fluid is injected into the eye, an abnormally high intra ocular pressure will result, leading to complications. Using the stereogrammetry program, the elevation of the edges of the sutured incision can be followed, allowing any incision gaps to be detected. These gaps indicate that intra ocular pressure is too high and accordingly, some saline should be removed.
In accordance with another embodiment of the invention, the template 500 may provide one or more suturing patterns 506 that can be superposed onto the displayed stereoscopic view pair. The suturing patterns 506 may be stored in memory, and may be operator selectable from a software menu using, for example, operator interface 209 described above. Once selected, the suturing pattern 506 may be superposed on the displayed stereoscopic view 501 and, if desired, further modified. Suturing patterns 506 selected by the viewer may be scalable and may be capable of orientation in any direction. In various embodiments, the viewer may be provided the capability to create new suturing patterns from scratch, which may then be saved into memory. Suturing patterns 506 may be displayed, without limitation, in various colors and/or as a wire frame image, such that the viewer can have a clear view of the subject being worked on. In various embodiments, patterns or markings other than suturing patterns may be superposed onto the displayed stereoscopic view pair, as desired. Markings may be, without limitation, alphanumeric, and/or geometric, such as lines, arrows, and circles, and may be used, for example, to label points of interest.
In other embodiments of the invention, the template 500 may show the position and extent of an incision to be made in a surgical procedure. Such types of incisions may include, for example, incisions for cataract removal and for repair of a cornea. A template may show the correct size and position for a capsularhexis procedure, for example, by superimposing a circle on the center of a cataractous lens. Since the capsularhexis is typically 5-6 mm in diameter, templates including circle of 5 mm, 5.5 mm and 6 mm are provided. In another embodiment, a template showing a triangle is provided for assisting a surgeon in removing a portion of the iris during an iridectomy. The size of the base of the triangle is approximately 0.5 to 2.0 mm with the base positoned at the iris edge. In a further embodiment of the invention, the template identifies retinal tissue for dissection.
Other templates that may be displayed show the major axes of astigmatism of the cornea and the optical center of the cornea. In a specific embodiment of the invention, a second template may be overlaid on the eye, showing the major axes of astigmatism of the cornea and the optical center of the cornea as the procedure progresses. The axes of astigmatism and center of the cornea may be determined, for example, by pattern recognition by the processor.
In other embodiments of the invention, the template 500 may provide guidance for placing an implant, including a transplant, in an eye. The template may, for example, identify the placement and shape of the implant, so that the surgeon may make an appropriately sized incision at the site. Further, once a site has been prepared for an implant or a transplant, the surgeon may identify with an input device the size and shape of the prepared site. A template may then be generated by the processor to match the outline of the prepared site. This template may be superimposed on the image of the eye at the exact site where the transplant tissue is to be harvested, as a guide to the surgeon. This technique will be useful, for example, in pterygium surgery where, after the removal of the pterygium, healthy conjunctival tissue is harvested and then sutured in place to cover the site. This technique will also be valuable, for example, for identifying a corneal transplant from a donor eye and, in particular, for displaying a further template that shows the suture pattern for the transplant.
In another embodiment of the invention, the template 500 may indicate the position and angle at which an instrument should be held during a part of a procedure. In certain surgical maneuvers, the angle that the surgeon holds an instrument, such as a scalpel, scissors or a heating probe, is crucial in creating a precise and reproducible effect.
In other embodiments of the invention, a variety of data related to a surgical procedure may be received by the system and displayed on the monitor 112 contemporaneously with display of images of eye structures. Such data may include data that is updated periodically and then displayed. Display of such real-time data may be on the same monitor as is used to display other information, such as, without limitation, images of an eye, measurements and instructions. The display of such data may also be in a window on the display surface. Thus, in some embodiments of the invention, the surgeon's attention may be directed to a single display surface or a small number of display surfaces for information, rather than being directed to multiple displays, gauges, and meters in various parts of the operating room. The data received by the system may be received through a variety of electronic interfaces, as are known in the art, and the data received may be manipulated by the processor 108 and may also be stored before the data are presented on a display surface. Data that may be received by the processor and displayed includes, without limitation, data related to a patient, such as a temperature, a refractive power, a wave front map, a topography of an eye structure, an intraocular pressure, an area of a tissue, a volume of a tissue, and a corneal thickness. Other data such as a blood pressure, a pulse, and an oxygen level may be received by the system and displayed. Additional data such as time of day, running time of a procedure or a portion of a procedure, or data describing parameters of ancillary equipments such as pumps and suction devices may be displayed. In a specific embodiment of the invention, data characterizing fluid flow and suction may be displayed.
In other various embodiments of the invention, the template 500 provides surgical guidance. Such guidance may be preprogrammed and/or interpretive. For example, processor 108 may include a collection of selectable videos and/or still images, with each video and/or still image pertaining, without limitation, to an anatomy of healthy or diseased tissue (such as, for example, lesions of the eye), a particular surgical procedure and/or a complication that may arise during surgery. The video(s) may be produced by experienced surgeons. The selected video 509 and/or still image may be displayed in the template 500. If a certain step in an operation needs to be reviewed or if a complication is encountered, the surgeon or other observer can temporarily pause the surgery being performed and play the appropriate portion of video 509 so as to obtain substantially instant expert advice. In other embodiments, the video 509 may be played simultaneously with the on-going surgery, with, for example, the surgeon stepping through the surgery based on instructions provided by the video. The video(s) and/or still images may be stored, without limitation, on tape, DVD, or other suitable recording mediums known in the art. Both graphical material and/or audio material may be presented. Operator interface 209 (see
In other embodiments of the invention, a plurality of images of an eye may be presented on the display monitor 112 simultaneously. One or more of these images may be retrieved from a storage device and displayed, for example. One or more of these images may be captured by video cameras positioned with various angles and placements with respect to the eye.
Other guidance provided by the template 500 may include, without limitation, medical diagnostic advice based on inputs from the surgeon and/or from information that processor 8 obtains directly from the stereoscopic view pair. For example, a surgeon may provide to the processor 8, via the operator interface 209, symptoms, key words, and/or questions. Processor's 8 responses to the query may involve various levels of search capability, artificial intelligence, interpretive capability, and/or pattern recognition capability, as known in the art. Using pattern recognition, the processor 8 may, for example, identify various structures and/or diseased parts. Alarms based on an occurrence of a triggering event may be provided. Such alarms may be graphical and/or audio.
Processor 8 may be connected to a network, which may be the Internet. Thus, for example, queries of the surgeon and/or various portions of the video signal representative of a stereoscopic view pair and/or template may be transmitted to various remote locations. Observers and/or processors at the remote location(s) may provide, without limitation, advice, diagnosis, tutorial, pictorial, audio, or video information to processor 8 that can then be presented to the surgeon and/or stored in memory. Communication via the network may occur in real-time, with minimal delay.
Various embodiments of the invention may be implemented as a computer program product for use with a computer system. Such implementation may include a series of computer instructions fixed either on a tangible medium, such as a computer readable media (e.g., a diskette, CD-ROM, ROM, or fixed disk), or fixed in a computer data signal embodied in a carrier wave that is transmittable to a computer system via a modem or other interface device, such as a communications adapter connected to a network over a medium. The medium may be either a tangible medium (e.g., optical or analog communications lines) or a medium implemented with wireless techniques (e.g., microwave, infrared or other transmission techniques). The series of computer instructions embodies all or part of the functionality previously described herein with respect to the system. Those skilled in the art should appreciate that such computer instructions can be written in a number of programming languages for use with many computer architectures or operating systems. Furthermore, such instructions may be stored in any memory device, such as semiconductor, magnetic, optical or other memory devices, and may be transmitted using any communications technology, such as optical, infrared, microwave, or other transmission technologies. It is expected that such a computer program product may be distributed as a removable medium with accompanying printed or electronic documentation (e.g., shrink wrapped software), preloaded with a computer system (e.g., on system ROM or fixed disk), or distributed from a server or electronic bulletin board over the network (e.g., the Internet or World Wide Web).
The present invention may be embodied in other specific forms without departing from the true scope of the invention, as defined by the appended claims. The described embodiments are to be considered in all respects only as illustrative and not restrictive.
This application is a continuation-in-part application of a U.S. patent application entitled “A System and Method of Providing Visual Documentation during Surgery,” Ser. No. 10/265,303, filed Oct. 4, 2002. This application also claims priority from both U.S. provisional patent application Ser. No. 60/327,323, filed Oct. 5, 2001, entitled “An Intelligent Ophthalmic Microsurgical System” and U.S. provisional patent application Ser. No. 60/348,545, filed Jan. 16, 2002, entitled “A Failsafe, Multiview, Electro-optical, Cluster for Ophthalmic Microsurgery.” Each of the patent applications described in this paragraph is hereby incorporated by reference in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 60327323 | Oct 2001 | US | |
| 60348545 | Jan 2002 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10265303 | Oct 2002 | US |
| Child | 10969226 | Oct 2004 | US |
