Patent Application
20230404400
The present disclosure relates generally to an ophthalmic observation apparatus, a method of controlling the same, a program, and a recording medium.
An ophthalmic observation apparatus is an apparatus for observing an eye of a patient (which will be referred to as a subject's eye hereinafter). Ophthalmic observation is conducted to grasp the condition of the subject's eye in various situations such as examination, surgery, and treatment.
Conventional ophthalmic observation apparatuses are configured to provide a user with a magnified image formed by an objective lens, a variable magnification optical system, etc. via an eyepiece. In recent years, some ophthalmic observation apparatuses are configured to photograph a magnified image formed by an objective lens, a variable magnification optical system, etc. with an image sensor, and display the photographed image obtained (such an ophthalmic observation apparatus will be referred to as a digital ophthalmic observation apparatus). Examples of such digital ophthalmic observation apparatuses include surgical microscopes, slit lamp microscopes, and fundus cameras (retinal cameras). In addition, various kinds of ophthalmic examination apparatuses such as refractometers, keratometers, tonometers, specular microscopes, wavefront analyzers, and microperimeters are also provided with the function of the digital ophthalmic observation apparatus.
Furthermore, some ophthalmic observation apparatuses of recent years use optical scanning (scanning-type ophthalmic observation apparatus). Examples of such ophthalmic observation apparatuses include scanning laser ophthalmoscopes (SLOs), and optical coherence tomography (OCT) apparatuses.
Generally, an ophthalmic observation apparatus is configured to provide a moving image of a subject's eye to a user (e.g., a health professional (health care practitioner) such as a doctor). A typical digital ophthalmic observation apparatus is configured to perform photographing of a moving image using infrared light and/or visible light as illumination light, and real-time display of the moving image obtained by the moving image photography. On the other hand, a typical scanning-type ophthalmic observation apparatus is configured to perform data collection (data acquisition) by repetitive optical scanning, real-time image reconstruction based on datasets sequentially collected, and real-time moving image display of images sequentially reconstructed. The real-time moving image provided in these ways is referred to as an observation image or a live image.
There are various methods or techniques of observing a subject's eye. A method or technique of observing a subject's eye is referred to as an observation mode. For example, Patent Document 1 below discloses an apparatus configured to be capable of performing selective application of various observation modes with individually different conditions such as illumination angles, magnifications, and light amounts. Further, Patent Document 2 below discloses an apparatus configured to be capable of linking an operation of changing an illumination angle and an operation of changing a light amount.
One of well-known observation modes is coaxial illumination (also referred to as 0-degree illumination). Coaxial illumination is an observation mode in which illumination light is projected onto the subject's eye along the optical axis (or along a direction that is slightly oblique with respect to the optical axis) of the observation optical system (or the photography optical system). Coaxial illumination is used to obtain a red reflex image (transillumination image) formed by utilizing diffuse reflection from eye fundus. A red reflex image is typically used for observation of opacity of ocular media (optic media) (e.g., crystalline lens, vitreous body, etc.), observation of the arrangement of an intraocular lens, and so forth.
In coaxial illumination, the area in which fundus reflection of the illumination light returns to the optical system is limited, which requires adjusting the position of the optical system so that a wider area can be observed more brightly. According to conventional technology, position adjustments of the optical system have been performed manually and therefore, the search for the optimum position of the optical system has been time consuming and labor intensive. In addition, there has existed no means to ascertain whether the optimum position has been actually achieved. For example, even if the optical system has been guided to a position considered optimal by manual operation, there still has remained a possibility that a preferable position has existed, but there has been no way of knowing this. Note that surgery can be performed if the optical system is located at a suitable position to some extent. However, there is room for improvement when the following issues are taken into consideration: improvement in the success rate of surgery; shortening of time required for surgery; decrease in the difficulty of surgery; and reduction in illumination light amount. Note also that the same problem arises even in the case where an illumination method other than coaxial illumination is employed because of the fact that the state and condition of observed images typically depend on the position of the optical system.
One object of the present disclosure is to provide a novel method or technique for facilitating ophthalmic observation.
Some aspect examples are an ophthalmic observation apparatus for observing a subject's eye that includes: a moving image generating unit configured to generate a moving image by illuminating and photographing the subject's eye; a movement mechanism configured to move the moving image generating unit; an analyzing processor configured to sequentially analyze a plurality of still images included in the moving image generated by the moving image generating unit being moved by the movement mechanism to sequentially detect images of a predetermined site of the subject's eye; and a first controller configured to control the movement mechanism based on a change in a predetermined image parameter of the images sequentially detected by the analyzing processor.
In the ophthalmic observation apparatus of some aspect examples, the first controller may be configured to control the movement mechanism to stop movement of the moving image generating unit when the image parameter of the images sequentially detected by the analyzing processor satisfies a first condition.
In the ophthalmic observation apparatus of some aspect examples, the image parameter may include brightness, and the first condition may include at least one of a condition that the brightness is maximum and a condition that the brightness is equal to or larger than a first threshold value.
In the ophthalmic observation apparatus of some aspect examples, the first controller may be configured to control the movement mechanism to start movement of the moving image generating unit when the image parameter of the images sequentially detected by the analyzing processor satisfies a second condition.
In the ophthalmic observation apparatus of some aspect examples, the image parameter may include brightness, and the second condition may include a condition that the brightness is smaller than a second threshold value.
In the ophthalmic observation apparatus of some aspect examples, the first controller may be configured to determine a movement direction of the moving image generating unit based on the change in the image parameter and perform a control of the movement mechanism based on the movement direction determined.
In the ophthalmic observation apparatus of some aspect examples, the moving image generating unit may be configured to generate a moving image of an anterior eye segment of the subject's eye, and the predetermined site may include a pupil.
In the ophthalmic observation apparatus of some aspect examples, the moving image generating unit may be configured to generate a moving image of a posterior eye segment of the subject's eye, and the predetermined site may include an optic nerve head.
In the ophthalmic observation apparatus of some aspect examples, the control of the movement mechanism based on the change in the image parameter performed by the first controller may be activated and inactivated.
The ophthalmic observation apparatus of some aspect examples may further include an informing unit configured to perform informing when the image parameter of the images sequentially detected by the analyzing processor satisfies a third condition.
The ophthalmic observation apparatus of some aspect examples may further include a response receiving unit configured to receive a response of the user to the informing, wherein the first controller may be configured to perform a control of the movement mechanism based on the response received by the response receiving unit.
The ophthalmic observation apparatus of some aspect examples may further include a determining processor configured to perform determination of whether or not to perform the control of the movement mechanism based on the change in the image parameter performed by the first controller.
In the ophthalmic observation apparatus of some aspect examples, the determining processor may be configured to perform the determination based on the moving image generated by the moving image generating unit.
In the ophthalmic observation apparatus of some aspect examples, the determining processor may be configured to determine whether or not a treatment on the subject's eye is being performed based on the moving image generated by the moving image generating unit, and determine not to perform the control of the movement mechanism when it is determined that the treatment is being performed.
The ophthalmic observation apparatus of some aspect examples may further include a second controller configured to perform at least one of informing, increasing in an illumination light amount by the moving image generating unit, increasing in a gain of an image sensor of the moving image generating unit, and changing of a determination condition regarding the image parameter when an optimum value of the image parameter acquired during the control of the movement mechanism based on the change in the image parameter satisfies a fourth condition.
The ophthalmic observation apparatus of some aspect examples may further include a third controller configured to decrease an illumination light amount by the moving image generating unit on condition that the image parameter is included in a predetermined range after the control of the movement mechanism based on the change in the image parameter.
In the ophthalmic observation apparatus of some aspect examples, the image parameter may include at least one of brightness, contrast, sharpness, and color tone.
Some aspect examples are a method of controlling an ophthalmic observation apparatus including an optical system for observing a subject's eye and a processor, the method including: causing the ophthalmic observation apparatus to perform movement of the optical system and generation of a moving image of the subject's eye by the optical system in parallel; causing the processor to sequentially analyze a plurality of still images included in the moving image to sequentially detect images of a predetermined site of the subject's eye; and causing the processor to perform movement of the optical system based on a change in a predetermined image parameter of the images sequentially detected.
Some aspect examples are a program configured to cause a computer to execute the method of some aspect examples.
Some aspect examples are a computer-readable non-transitory recording medium storing the program of some aspect examples.
These aspect examples allow ophthalmic observation to be facilitated.
Some aspect examples of an ophthalmic observation apparatus, a method of controlling the same, a program, and a recording medium according to some embodiments will be described in detail with reference to the drawings. It should be noted that any of the matters and items described in the documents cited in the present disclosure and any known techniques and technologies may be combined with any of the aspect examples.
It should be noted that there exists a conventional technique or technology in which an optical system is devised for the purpose of improving the illumination performance of an ophthalmic observation apparatus (e.g., the performance of coaxial illumination). Even with products to which such a conventional technique or technology is applied however, it is still possible to make a further improvement in the illumination performance by combining the technique or technology of the present disclosure.
The ophthalmic observation apparatus according to some aspect examples is used in medical practice (healthcare practice) such as examination, surgery, and treatment of the subject's eye, in order to grasp (understand, recognize, find) the state of the subject's eye. The ophthalmic observation apparatus of the aspect examples described herein is mainly a surgical microscope system. However, ophthalmic observation apparatuses of embodiments are not limited to surgical microscope systems. For example, the ophthalmic observation apparatus of some aspect examples may be any of a slit lamp microscope, a fundus camera, a refractometer, a keratometer, a tonometer, a specular microscope, a wavefront analyzer, a microperimeter, an SLO, and an OCT apparatus. Also, the ophthalmic observation apparatus of some aspect examples may be a system that includes any one or more of these apparatus examples. In a wider sense, the ophthalmic observation apparatus of some aspect examples may be any type of ophthalmic apparatus having an observation function.
A target ocular site for observation (ocular site to be observed, ocular site subject to observation) by using the ophthalmic observation apparatus may be any site of the subject's eye, and may be any site of the anterior eye segment and/or any site of the posterior eye segment. Examples of the observation target sites of the anterior eye segment include cornea, iris, anterior chamber, corner angle, crystalline lens, ciliary body, and zonule of Zinn. Examples of the observation target sites of the posterior eye segment include retina, choroid, sclera, and vitreous body. The observation target site is not limited to tissues of an eye ball, and may be any site subject to be observed in ophthalmic medical practice (and/or medical practice in other medical fields) such as eyelid, meibomian gland, and orbit (eye socket, eye pit).
At least one or more of the functions of the elements described in the present disclosure are implemented by using a circuit configuration (or circuitry) or a processing circuit configuration (or processing circuitry). The circuitry or the processing circuitry includes any of the followings, all of which are configured and/or programmed to execute at least one or more functions disclosed herein: a general purpose processor, a dedicated processor, an integrated circuit, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), or a field programmable gate array (FPGA)), a conventional circuit configuration or circuitry, and any combination of these. A processor is considered to be processing circuitry or circuitry that includes a transistor and/or another circuitry. In the present disclosure, circuitry, a unit, a means, or a term similar to these is hardware that executes at least one or more functions disclosed herein, or hardware that is programmed to execute at least one or more functions disclosed herein. Hardware may be the hardware disclosed herein, or alternatively, known hardware that is programmed and/or configured to execute at least one or more functions described herein. In the case where the hardware is a processor, which may be considered as a certain type of circuitry, then circuitry, a unit, a means, or a term similar to these is a combination of hardware and software. In this case, the software is used to configure the hardware and/or the processor.
The ophthalmic observation apparatus 1 (surgical microscope system, operation microscope system) according to the present embodiment includes the operation device 2, the display device 3, and the surgical microscope (operation microscope) 10. In some aspects, the surgical microscope 10 may include at least one of the operation device 2 and the display device 3. In some aspects, the display device 3 may not be included in the ophthalmic observation apparatus 1. In other words, the display device 3 may be a peripheral device of the ophthalmic observation apparatus 1.
The operation device 2 includes an operation device and/or an input device. For example, the operation device 2 may include any of a button, a switch, a mouse, a keyboard, a trackball, an operation panel, a dial, and so forth. Typically, the operation device 2 includes a foot switch, like standard (general, normal, usual) ophthalmic surgical microscopes. Further, the operation device 2 may also be configured in such a manner that the user performs operations using voice recognition, line-of-sight (gaze) input, or like input technologies.
The display device 3 displays an image of the subject's eye acquired by the surgical microscope 10. The display device 3 includes a display device such as a flat panel display. The display device 3 may include any of various kinds of display devices such as a touch panel. The display device 3 of some typical aspects includes a display device with a large screen. The display device 3 includes one or more display devices. In the case where the display device 3 includes two or more display devices, for example, one may be a display device with a relatively large screen and one of the other(s) may be a display device with a relatively small screen. Also, a configuration may be employed in which a plurality of display regions is provided in one display device to display a plurality of pieces of information.
The operation device 2 and the display device 3 do not have to be separate devices. For example, a device having both the operation function and the display function, such as a touch panel, may be used as the display device 3. In such a case, the operation device 2 may include a computer program in addition to the touch panel. A content of an operation made by the operation device 2 is sent to a processor (not shown in the drawings) as an electric signal. Further, a graphical user interface (GUI) displayed on the display device 3 and the operation device 2 may be used to conduct operations (instructions) and input information. In some aspects, the functions of the operation device 2 and the display device 3 may be implemented with a touch screen.
The surgical microscope 10 is used for observation of the eye of a patient (subject's eye) in the supine position. The surgical microscope 10 performs photographing of the subject's eye to generate digital image data. In particular, the surgical microscope 10 generates a moving image of the subject's eye. The moving image (video, movie) generated by the surgical microscope 10 is transmitted to the display device 3 through a wired and/or wireless signal path and displayed on the display device 3. The user (e.g., surgeon) can carry out surgery while observing the subject's eye through the displayed image. In addition to such observation through the displayed image, the surgical microscope 10 of some aspects may also be capable of providing observation through an eyepiece as in conventional technology.
In some aspects, the surgical microscope 10 includes a communication device for transmitting and receiving electrical signals to and from the operation device 2. The operation device 2 receives an operation (instruction) performed by the user and generates an electric signal (operation signal) corresponding to the operation. The operation signal is transmitted to the surgical microscope 10 through a wired and/or wireless signal path. The surgical microscope 10 executes processing corresponding to the operation signal received.
An example of the configuration of the optical system of the surgical microscope 10 will be described. Below, directions are defined as follows, for convenience of description: the z direction is defined to be the optical axis direction (direction along the optical axis) of the objective lens (the z direction is, for example, the vertical direction, the up and down direction during surgery); the x direction is defined to be a predetermined direction perpendicular to the z direction (the x direction is, for example, the horizontal direction during surgery, and the left and right direction for the surgeon and the patient during surgery); and the y direction is defined to be the direction perpendicular to both the z and x directions (the y direction is, for example, the horizontal direction during surgery, the front and back direction for the surgeon during surgery, and the body axis direction (direction along the body axis) for the patient during surgery).
In addition, the case where the observation optical system includes a pair of left and right optical systems (optical systems capable of binocular observation) will be mainly described below. However, an observation optical system of some other aspects may have an optical system for monocular observation, and it will be understood by those skilled in the art that the configuration described below may be incorporated into the aspects for monocular observation.
The surgical microscope 10 includes the objective lens 20, the dichroic mirror DM1, the illumination optical system 30, and the observation optical system 40. The observation optical system 40 includes the zoom expander 50, and the imaging camera 60. In some aspects, the illumination optical system 30 or the observation optical system 40 includes the dichroic mirror DM1.
The objective lens 20 is arranged to face the subject's eye. The objective lens 20 is arranged such that its optical axis is oriented along the z direction. The objective lens 20 may include two or more lenses.
The dichroic mirror DM1 couples the optical path of the illumination optical system 30 and the optical path of the observation optical system 40 with each other. The dichroic mirror DM1 is arranged between the illumination optical system 30 and the objective lens 20. The dichroic mirror DM1 transmits illumination light from the illumination optical system 30 and directs the illumination light to the subject's eye through the objective lens 20. Also, the dichroic mirror DM1 reflects return light from the subject's eye incident through the objective lens 20 and directs the return light to the imaging camera 60 of the observation optical system 40.
The dichroic mirror DM1 coaxially couples the optical path of the illumination optical system 30 and the optical path of the observation optical system 40 with each other. In other words, the optical axis of the illumination optical system 30 and the optical axis of the observation optical system 40 intersect at the dichroic mirror DM1. In the case where the illumination optical system 30 includes an illumination optical system for left eye (31L) and an illumination optical system for right eye (31R) and where the observation optical system 40 includes an observation optical system for left eye 40L and an observation optical system for right eye 40R, the dichroic mirror DM1 coaxially couples the optical path of the illumination optical system for left eye (the first illumination optical system 31L) and the optical path of the observation optical system for left eye 40L with each other, and coaxially couples the optical path of the illumination optical system for right eye (the first illumination optical system 31R) and the optical path of the observation optical system for right eye 40R with each other.
The illumination optical system 30 is an optical system for illuminating the subject's eye through the objective lens 20. The illumination optical system 30 may be configured to selectively illuminate the subject's eye with two or more pieces of illumination light having different color temperatures. The illumination optical system 30 projects illumination light having a designated color temperature onto the subject's eye under the control of a controller (the controller 200 described later).
The illumination optical system 30 includes the first illumination optical systems 31L and 31R and the second illumination optical system 32.
Each of the optical axis OL of the first illumination optical system 31L and the optical axis OR of the first illumination optical system 31R is arranged with the optical axis of the objective lens 20 in a substantially coaxial manner. Such arrangements enable a coaxial illumination mode and therefore make it possible to obtain a red reflex image (transillumination image) formed by utilizing diffuse reflection from eye fundus. The present aspect allows the red reflex image of the subject's eye to be observed with both eyes.
The second illumination optical system 32 is arranged in such a manner that its optical axis OS is eccentric (deviated, shifted) from the optical axis of the objective lens 20. The first illumination optical systems 31L and 31R and the second illumination optical system 32 are arranged such that the deviation of the optical axis OS with respect to the optical axis of the objective lens 20 is larger than the deviations of the optical axes OL and OR with respect to the optical axis of the objective lens 20. Such arrangements enable an illumination mode referred to as “angled illumination (oblique illumination)” and therefore enables binocular observation of the subject's eye while preventing ghosting caused by corneal reflection or the like. In addition, the arrangements enable detailed observation of unevenness and irregularities of sites and tissues of the subject's eye.
The first illumination optical system 31L includes the light source 31LA and the condenser lens 31LB. The light source 31LA outputs illumination light having a wavelength in the visible range (visible region) corresponding to color temperature of 3000 K (kelvins), for example. The illumination light emitted from the light source 31LA passes through the condenser lens 31LB, passes through the dichroic mirror DM1, passes through the objective lens 20, and then is incident on the subject's eye.
The first illumination optical system 31R includes the light source 31RA and the condenser lens 31RB. The light source 31RA also outputs illumination light having a wavelength in the visible range corresponding to color temperature of 3000 K, for example. The illumination light emitted from the light source 31RA passes through the condenser lens 31RB, passes through the dichroic mirror DM1, passes through the objective lens 20, and then is incident on the subject's eye.
The second illumination optical system 32 includes the light source 32A and the condenser lens 32B. The light source 32A outputs illumination light having a wavelength in the visible range corresponding to a color temperature within the range of 4000 K to 6000 K, for example. The illumination light emitted from the light source 32A passes through the condenser lens 32B, passes through the objective lens 20 without passing through the dichroic mirror DM1, and then is incident on the subject's eye.
In the present aspect example, the color temperature of the illumination light from the first illumination optical systems 31L and 31R is lower than the color temperature of the illumination light from the second illumination optical system 32. Such a configuration makes it possible to observe the subject's eye in warm colors using the first illumination optical systems 31L and 31R, and therefore enables detailed observation of the structure and morphology of the subject's eye.
In some aspects, each of the optical axes OL and OR is movable relative to the optical axis of the objective lens 20. The direction of the relative movement is a direction that intersects the optical axis of the objective lens 20, and the relative movement is represented by a displacement vector in which at least one of the x component and the y component is not zero. In some aspects, the optical axes OL and OR may be mutually independently movable. On the other hand, in some aspects, the optical axes OL and OR may be integrally movable. For example, the surgical microscope 10 includes a movement mechanism (31d) configured to move the first illumination optical systems 31L and 31R mutually independently or integrally, and therefore the movement mechanism moves the first illumination optical systems 31L and 31R mutually independently or integrally in a direction intersecting the optical axis of the objective lens 20. Such a configuration makes it possible to conduct adjustment of the appearance condition (appearance state) of the subject's eye. In some aspects, the movement mechanism operates under the control of a controller (the controller 200 described later).
In some aspects, the optical axis OS is movable relative to the optical axis of the objective lens 20. The direction of the relative movement is a direction that intersects the optical axis of the objective lens 20, and the relative movement is represented by a displacement vector in which at least one of the x component and the y component is not zero. For example, the surgical microscope 10 includes a movement mechanism (32d) configured to move the second illumination optical system 32, and therefore the movement mechanism moves the second illumination optical system 32 in a direction that intersects the optical axis of the objective lens 20. With such a configuration, it becomes possible to conduct adjustment of the appearance condition (appearance state) of unevenness and irregularities of sites and tissues of the subject's eye. In some aspects, the movement mechanism operates under the control of a controller (the controller 200 described later).
As described above, the present aspect is configured such that the illumination optical system 30 is arranged at the position directly above the objective lens 20 (the position in the transmission direction of the dichroic mirror DM1) and the observation optical system 40 is arranged at the position in the reflection direction of the dichroic mirror DM1. For example, the observation optical system 40 may be arranged in such a manner that the angle formed by the optical axis of the observation optical system 40 and the plane perpendicular to the optical axis of the objective lens 20 (the xy plane) belongs to the range between −20 degrees and +20 degrees.
According to the configuration of the present aspect, the observation optical system 40, which typically has a longer optical path length than the illumination optical system 30, is arranged substantially parallel to the xy plane. Hence, the observation optical system 40 of the present aspect does not interfere with the surgeon's field of view while conventional surgical microscopes, whose observation optical system is oriented along the vertical direction in front of the surgeon's eyes, do. Therefore, the surgeon is capable of easily seeing the screen of the display device 3 arranged in front of the surgeon. In other words, the visibility of displayed information (images and videos of the subject's eye, and other various kinds of reference information) during surgery etc. is improved. In addition, since the housing is not placed in front of the surgeon's eyes, it does not give a sense of oppression to the surgeon, thereby reducing the burden on the surgeon.
The observation optical system 40 is an optical system for observation of an image formed based on return light of the illumination light incident from the subject's eye through the objective lens 20. In the present aspect, the observation optical system 40 guides the image to an image sensor of the imaging camera 60.
As described above, the observation optical system 40 includes the observation optical system for left eye 40L and the observation optical system for right eye 40R. The configuration of the observation optical system for left eye 40L and the configuration of the observation optical system for right eye 40R are the same as or similar to one another. In some aspects, the observation optical system for left eye 40L and the observation optical system for right eye 40R may be configured in such a manner that their optical arrangements can be changed independently of each other.
The zoom expander 50 is also referred to as a beam expander, a variable beam expander, or the like. The zoom expander 50 includes the zoom expander for left eye 50L and the zoom expander for right eye 50R. The configuration of the zoom expander for left eye 50L and the configuration of the zoom expander for right eye 50R are the same as or similar to each other. In some aspects, the zoom expander for left eye 50L and the zoom expander for right eye 50R may be configured in such a manner that their optical arrangements can be changed independently of each other.
The zoom expander for left eye 50L includes the plurality of zoom lenses 51L, 52L, and 53L. At least one of the zoom lenses 51L, 52L, and 53L is movable in the direction along the optical axis with a variable magnification mechanism (not shown in the drawings).
Similarly, the zoom expander for right eye 50R includes the plurality of zoom lenses 51R, 52R, and 53R, and at least one of the zoom lenses 51R, 52R, and 53R is movable in the direction along the optical axis with a variable magnification mechanism (not shown in the drawings).
The variable magnification mechanism(s) may be configured to move a zoom lens of the zoom expander for left eye 50L and a zoom lens of the zoom expander for right eye 50R mutually independently or integrally in the directions along the optical axes. As a result of this, the magnification ratio for photographing the subject's eye is changed. In some aspects, the variable magnification mechanism(s) operates under the control of a controller (the controller 200 described later).
The imaging camera 60 is a device that photographs an image formed by the observation optical system 40 and generates digital image data. The imaging camera 60 is typically a digital camera (digital video camera). The imaging camera 60 includes the imaging camera for left eye 60L and the imaging camera for right eye 60R. The configuration of the imaging camera for left eye 60L and the configuration of the imaging camera for right eye 60R are the same as or similar to one another. In some aspects, the imaging camera for left eye 60L and the imaging camera for right eye 60R may be configured such that their optical arrangements can be changed independently of each other.
The imaging camera for left eye 60L includes the imaging lens 61L and the image sensor 62L. The imaging lens 61L forms an image based on the return light that has passed through the zoom expander for left eye 50L, on the imaging surface (light receiving surface) of the image sensor 62L. The image sensor 62L is an area sensor, and may typically be a charge-coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor. The image sensor 62L operates under the control of a controller (the controller 200 described later).
The imaging camera for right eye 60R includes the imaging lens 61R and the image sensor 62R. The imaging lens 61R forms an image based on the return light that has passed through the zoom expander for right eye 50R, on the imaging surface (light receiving surface) of the image sensor 62R. The image sensor 62R is an area sensor, and may typically be a CCD image sensor or a CMOS image sensor. The image sensor 62R operates under the control of a controller (the controller 200 described later).
The processing system of the ophthalmic observation apparatus 1 will be described. Some configuration examples of the processing system are shown in
The controller 200 executes a control of each part of the ophthalmic observation apparatus 1. The controller 200 includes the main controller 201 and the memory 202. The main controller 201 includes a processor and executes a control of each part of the ophthalmic observation apparatus 1. For example, the processor may load and run a program stored in the memory 202 or another storage device, thereby implementing a function according to the present aspect. In addition, the processor may use (e.g., referring, processing, calculating, etc.) data and/or information stored in the memory 202 or another storage device in order to implement a function according to the present aspect.
The main controller 201 may control the light sources 31LA, 31RA, and 32A of the illumination optical system 30, the image sensors 62L and 62R of the observation optical system 40, the movement mechanisms 31d and 32d, the variable magnification mechanisms 50Ld and 50Rd, the operation device 2, the display device 3, and other component parts.
Controls of the light source 31LA include turning on and off the light source, adjusting the light amount, adjusting the diaphragm (aperture), and so forth. Controls of the light source 31RA include turning on and off the light source, adjusting the light amount, adjusting the diaphragm (aperture), and so forth. The main controller 201 may perform mutually exclusive controls of the light sources 31LA and 31RA. Controls of the light source 32A include turning on and off the light source, adjusting the light amount, adjusting the diaphragm (aperture), and so forth.
In the case where the illumination optical system 30 includes a light source whose color temperature can be varied, the main controller 201 may change the color temperature of emitted illumination light by controlling such a light source.
Controls of the image sensor 62L include exposure adjustment, gain adjustment, photographing rate adjustment, and so forth. Controls of the image sensor 62R include exposure adjustment, gain adjustment, photographing rate adjustment, and so forth. Further, the main controller 201 may control the image sensors 62L and 62R in such a manner that the photographing timings of the image sensors 62L and 62R match each other, or in such a manner that the difference between the photographing timings of the image sensors 62L and 62R lies within a predetermined time. In addition, the main controller 201 may perform a control of loading digital data obtained by the image sensors 62L and 62R.
The movement mechanism 31d moves the light sources 31LA and 31RA mutually independently or integrally in a direction that intersects the optical axis of the objective lens 20. By controlling the movement mechanism 31d, the main controller 201 moves the optical axes OL and OR mutually independently or integrally with respect to the optical axis of the objective lens 20.
The movement mechanism 32d moves the light source 32A in a direction that intersects the optical axis of the objective lens 20. By controlling the movement mechanism 32d, the main controller 201 moves the optical axis OS with respect to the optical axis of the objective lens 20.
The movement mechanism 70 moves the surgical microscope 10. For example, the movement mechanism 70 is configured to integrally move at least part of the illumination optical system 30 and the observation optical system 40. This configuration makes it possible to change the relative positions of the at least part of the illumination optical system 30 and the observation optical system 40 with respect to the subject's eye while maintaining the relative positional relationship between at least part of the illumination optical system 30 and the observation optical system 40. In some aspects, the movement mechanism 70 is configured to integrally move the first illumination optical systems 31L and 31R and the observation optical system 40. With this, the relative positions of the first illumination optical systems 31L and 31R with respect to the subject's eye and the relative position of the observation optical system 40 with respect to the subject's eye can be changed while maintaining the state (condition) of coaxial illumination. In some aspects, the movement mechanism 70 is configured to integrally move the second illumination optical system 32 and the observation optical system 40. With this, the relative positions of the second illumination optical system 32 and the observation optical system 40 with respect to the subject's eye can be changed while maintaining the illumination angle for oblique illumination. In some aspects, the movement mechanism 70 is configured to integrally move the first illumination optical systems 31L and 31R, the second illumination optical system 32, and the observation optical system 40. This makes it possible to change the relative positions of the illumination optical system 30 and the observation optical system 40 with respect to the subject's eye while maintaining both the state (condition) of coaxial illumination and the illumination angle for oblique illumination. The movement mechanism 70 operates under a control of the controller 200.
In some aspects, the main controller 201 may be configured to control at least two of the movement mechanisms 31d, 32d, and 70 in an interlocking manner.
The variable magnification mechanism 50Ld moves at least one of the plurality of zoom lenses 51L to 53L of the zoom expander for left eye 50L in the optical axis direction (direction along the optical axis). The main controller 201 changes the magnification ratio of the observation optical system for left eye 40L by controlling the variable magnification mechanism 50Ld.
Similarly, the variable magnification mechanism 50Rd moves at least one of the plurality of zoom lenses 51R to 53R of the zoom expander for right eye 50R in the optical axis direction (direction along the optical axis). The main controller 201 changes the magnification ratio of the observation optical system for right eye 40R by controlling the variable magnification mechanism 50Rd.
Controls for the operation device 2 include an operation permission control, an operation prohibition control, an operation signal transmission control and/or an operation signal reception control from the operation device 2, and other controls. The main controller 201 receives an operation signal generated by the operation device 2 and executes a control corresponding to the operation signal received.
Controls for the display device 3 include an information display control and other controls. As a display controller, the main controller 201 displays an image based on digital image data generated by the image sensors 62L and 62R on the display device 3. Typically, the main controller 201 may display a moving image (video, movie) based on digital image data (video signal) generated by the image sensors 62L and 62R on the display device 3. Further, the main controller 201 may display a still image (frame) included in the moving image on the display device 3. In addition, the main controller 201 may display an image (a moving image, a still image, etc.) obtained by processing the digital image data generated by the image sensors 62L and 62R on the display device 3. Furthermore, the main controller 201 may display, on the display device 3, any information generated by the ophthalmic observation apparatus 1, any information acquired from the outside by the ophthalmic observation apparatus 1, and other types of information.
Further, as a display controller, the main controller 201 may create an image for left eye from the digital image data generated by the image sensor 62L and create an image for right eye from the digital image data generated by the image sensor 62R, and then display the created image for left eye and the created image for right eye on the display device 3 in such a manner as to enable stereoscopic vision. For example, the main controller 201 may create a pair of left and right parallax images from the image for left eye and the image for right eye, and display the pair of parallax images on the display device 3. With this, the user (e.g., surgeon) can recognize the pair of parallax images as a stereoscopic image by using a known stereoscopic method or technique. The stereoscopic method applicable to the present aspect may be freely selected, and for example, may be any of the following methods: a stereoscopic method for naked eyes; a stereoscopic method using an auxiliary device (polarized glasses, etc.); a stereoscopic method by applying image processing (image synthesis, image composition, rendering, etc.) to an image for left eye and an image for right eye; a stereoscopic method by displaying a pair of parallax images simultaneously; a stereoscopic method by alternately displaying a pair of parallax images; and a stereoscopic method of a combination of two or more of the above methods.
The data processor 210 executes various kinds of data processes. Some examples of processing that may be executed by the data processor 210 will be described below. The data processor 210 (each element thereof) includes a processor that operates on the basis of predetermined software (program), and is implemented by the cooperation of hardware and software.
Some examples of processing that may be executed by the data processor 210 will be described together with related elements.
The controller 200 of the present aspect may be configured to perform the following controls in parallel which is rereferred to as a parallel control: a control of the movement mechanism 70 to integrally move the illumination optical system 30 and the observation optical system 40, which is referred to as a movement control; and a control of the illumination optical system 30 and the observation optical system 40 to perform moving image photography (including illumination and photography) of the subject's eye, which is referred to as a photography control. The mode or aspect of the parallel control may be freely designed or determined. For example, the parallel control may include a simultaneous execution of the movement control and the photography control (referred to as a simultaneous control), an alternate execution of the movement control and the photography control (referred to as an alternate control), or a combination of the simultaneous control and the alternate control.
One or more frames (still images) of a moving image (frame group, still image group) acquired by the parallel control performed in this way, are input into the analyzing processor 211. Here, all the still images acquired by the parallel control may be input into the analyzing processor 211, or only still images selected by thinning or like processing may be input into the analyzing processor 211. In the latter case, for example, the still image selection processing is executed by the controller 200 or the data processor 210A. It should be noted that in the former case, the analyzing processor 211 may perform the still image selection processing. In the present aspect, the parallel control and processing performed by the analyzing processor 211 are executed both sequentially and in real time. Therefore, overall processing load can be reduced by assigning the still image selection processing to the analyzing processor 211. Note that the method for reducing processing load is not limited to the still image selection processing. In the case where the ophthalmic observation apparatus 1 has sufficient processing resources, no processing load reduction method may be employed.
The analyzing processor 211 is configured to sequentially analyze a plurality of still images included in a moving image generated by the photography control executed in parallel with the movement control, to sequentially detect images of a predetermined site of the subject's eye.
The site of the subject's eye detected by the analyzing processor 211 may be freely selected or determined. In the case where a target site (a part or region of the subject's eye) to be observed by the ophthalmic observation apparatus 1 is the anterior eye segment, the analyzing processor 211 may be configured to detect any of the following sites, for example: the pupil (its entirety, or a feature site such as the pupil edge, the pupil center, or the pupil center of gravity), the cornea (its entirety, or a feature site such as the corneal ring, the corneal edge, the corneal center, or the corneal apex), the iris (its entirety, or a feature site such as the iris inner edge, the iris outer edge, or the iris pattern), the anterior chamber (its entirety, or a feature site such as the anterior border, or the posterior border), the corner angle (its entirety, a peripheral site, etc.), the crystalline lens (its entirety, or a feature site such as the lens capsule, the anterior capsule, the posterior capsule, or the lens nucleus), the ciliary body, the zonule of Zinn, a blood vessel, and a lesion. In the case where a target site to be observed by the ophthalmic observation apparatus 1 is the posterior eye segment, the analyzing processor 211 may be configured to detect any of the following sites, for example: the optic nerve head, the macula, a blood vessel, the retina (its entirety, the surface, or one or more sub-tissues), the choroid (its entirety, the anterior surface, the posterior surface, or one or more sub-tissues), the sclera (its entirety, the anterior surface, the posterior surface, or one or more sub-tissues), the vitreous body (its entirety, an opaque region, a floating object (floater), a detached tissue, etc.), and a lesion. In the case where a target site to be observed by the ophthalmic observation apparatus 1 is not a tissue of an eye ball, the analyzing processor 211 may be configured to detect a freely selected site or tissue such as the eyelid, the meibomian glands, or the orbit (eye socket, eye pit). The site to be detected by the analyzing processor 211 may be selected or determined depending on an illumination method employed, a site subject to surgery, a surgical method conducted, or other factors.
The analyzing processor 211 may be configured to detect an image of a predetermined site of the subject's eye from a still image using a freely selected region extraction method or technique. In some examples of detecting an image of a site characterized by its brightness (an image of a site that has a distinctive feature in brightness), the analyzing processor 211 may be configured to perform detection of an image of this site of the subject's eye from a still image using brightness thresholding such as binarization. In the case of detecting an image of a site characterized by its shape (an image of a site that has a distinctive feature in shape), the analyzing processor 211 may be configured to perform detection of an image of this site of the subject's eye from a still image using shape analysis processing such as pattern matching. In the case of detecting an image of a site characterized by its color tone (an image of a site that has a distinctive feature in color tone), the analyzing processor 211 may be configured to perform detection of an image of this site of the subject's eye from a still image using color analysis processing such as feature color extraction. In some examples, the analyzing processor 211 may be configured to detect an image of a predetermined site of the subject's eye by applying segmentation to a still image to identify an image of this site of the subject's eye. Typically, segmentation is image processing for identifying a partial region (subregion) in a given image. Segmentation may include any known image processing technique, and some examples of which may include image processing such as edge detection and/or machine learning (e.g., deep learning) based segmentation.
The controller 200 (and the data processor 210A) is configured to control the movement mechanism 70 based on a change in a predetermined image parameter of the images of the predetermined site sequentially detected from the moving image by the analyzing processor 211. The controller 200 (and the data processor 210A) that performs this control corresponds to the first controller. The movement direction of the illumination optical system 30 and the observation optical system 40 moved by this control of the movement mechanism 70 may be freely selected or determined. For example, the movement direction may be any one or more of the vertical direction (the Z direction), the horizontal direction (the X direction), and the front and back direction (the Y direction).
As a result, the ophthalmic observation apparatus 1 becomes capable of performing position adjustment of the surgical microscope 10 according to a change in the image parameter in response to a change in the position of the surgical microscope 10 with respect to the subject's eye. For example, as mentioned above, in coaxial illumination, the area in which fundus reflection of the illumination light returns to the optical system is limited, which requires adjusting the position of the optical system so that a wider area can be observed more brightly. Since conventional technology has required to manually perform position adjustment of the optical system, the search for an optimum position of the optical system has been time consuming and labor intensive. In contrast, the present aspect is capable of automatically performing the search for an optimum position of the optical system while monitoring the image parameter. Furthermore, the present aspect is capable of, by referring to the image parameter, ascertaining whether the optimum position is actually achieved. Thus, according to the present aspect, ophthalmic observation can be facilitated, and also improvements can be made in various areas such as the success rate of surgery, the duration of time required for surgery, difficulty of surgery, and so forth. The same applies to the case where an illumination method other than coaxial illumination, such as angled illumination (oblique illumination), is employed.
The type or kind of the image parameter taken into consideration by the controller 200 (and the data processor 210A) may be freely selected or determined. For example, the image parameter may be at least one of brightness, contrast, sharpness, and color tone.
The controller 200 (and the data processor 210A) may be configured to control the movement mechanism 70 to stop movement of the illumination optical system 30 and the observation optical system 40 under the condition that the image parameter of the images of the predetermined site sequentially detected from the moving image by the analyzing processor 211 satisfies the first condition. In some examples, the controller 200 (and the data processor 210A) may be configured to control the movement mechanism 70 to stop movement of the illumination optical system 30 and the observation optical system 40 when the brightness (or, contrast or sharpness) of the images of the predetermined site sequentially detected from the moving image by the analyzing processor 211 reaches the maximum. In some other examples, the controller 200 (and the data processor 210A) may be configured to control the movement mechanism 70 to stop movement of the illumination optical system 30 and the observation optical system 40 when the brightness (or, contrast or sharpness) of the images of the predetermined site sequentially detected from the moving image by the analyzing processor 211 reaches or exceeds a predetermined threshold value. In some still other examples, the controller 200 (and the data processor 210A) may be configured to control the movement mechanism 70 to stop movement of the illumination optical system 30 and the observation optical system 40 when the color tone of the images of the predetermined site sequentially detected from the moving image by the analyzing processor 211 falls within a predetermined allowable range. The threshold value and the range in the first condition may be fixed or variable.
As a result of this, the illumination optical system 30 and the observation optical system 40 can be automatically guided and placed at positions where a suitable image parameter can be obtained. With this, advantageous effects such as facilitation of ophthalmic observation can be achieved.
The controller 200 (and the data processor 210A) may be configured to control the movement mechanism 70 to start movement of the illumination optical system 30 and the observation optical system 40 under the condition that the image parameter of the images of the predetermined site sequentially detected from the moving image by the analyzing processor 211 satisfies the second condition. In some examples, the controller 200 (and the data processor 210A) may be configured to control the movement mechanism 70 to start movement of the illumination optical system 30 and the observation optical system 40 when the brightness (or, contrast or sharpness) of the images of the predetermined site sequentially detected from the moving image by the analyzing processor 211 becomes less than a predetermined threshold value. In some other examples, the controller 200 (and the data processor 210A) may be configured to control the movement mechanism 70 to start movement of the illumination optical system 30 and the observation optical system 40 when the color tone of the images of the predetermined site sequentially detected from the moving image by the analyzing processor 211 falls outside a predetermined allowable range. The threshold value and the range in the second condition may be fixed or variable. It should be noted that the second condition may be the same as or different from the first condition.
As a result of this, the movement of the illumination optical system 30 and the observation optical system 40 can be automatically started in response to deterioration in the image parameter, and then an optimum position of the optical system can be searched. Therefore, it becomes possible to maintain the quality of the observation image. For example, the positional relationship between the subject's eye and the surgical microscope 10 may change due to an eye movement or any other causes and the image parameter may deteriorate after the optical system has been placed at an optimum position by automatic search. In such a case, according to the present aspect, the ophthalmic observation apparatus 1 is capable of automatically detecting the deterioration in the image parameter and then resuming the search for an optimum position of the optical system. This makes it possible to achieve advantageous effects such as facilitation of ophthalmic observation.
The controller 200 (and the data processor 210A) may be configured to determine a movement direction of the illumination optical system 30 and the observation optical system 40 based on a change in the image parameter. For example, the controller 200 (and the data processor 210A) determines whether or not a change in the image parameter caused by movement of the illumination optical system 30 and the observation optical system 40 in a certain direction is a favorable change. If the change in the image parameter is determined to be a favorable change, then the controller 200 (and the data processor 210A) determines this direction (the above certain direction) as a movement direction. On the other hand, if the change in the image parameter is determined to be an unfavorable change, then the controller 200 (and the data processor 210A) determines a direction other than this direction (the above certain direction), such as the direction opposite from this direction, as a movement direction. For example, the controller 200 (and the data processor 210A) determines whether or not a change in the image parameter caused by movement of the illumination optical system 30 and the observation optical system 40 in a certain direction is a favorable change. If the change in the image parameter is determined to be a favorable change, then the controller 200 (and the data processor 210A) determines this direction (the above certain direction) as a movement direction. On the other hand, if the change in the image parameter is determined to be an unfavorable change, then the controller 200 (and the data processor 210A) determines a direction other than this direction (the above certain direction), such as the direction opposite from this direction, as a movement direction. The controller 200 performs a control of the movement mechanism 70 based on the movement direction determined. As a specific example, if the brightness increases with the movement of the illumination optical system 30 and the observation optical system 40 in a certain direction, then the controller 200 (and the data processor 210A) can continue to move the illumination optical system 30 and the observation optical system 40 in this direction. If the brightness decreases with the movement in the certain direction, on the other hand, the controller 200 (and the data processor 210A) can switch the movement direction of the illumination optical system 30 and the observation optical system 40 from the certain direction to the opposite direction therefrom.
With this, the ophthalmic observation apparatus 1 is capable of automatically determining a movement direction of the surgical microscope 10 in accordance with a change in the image parameter, making it possible to achieve advantageous effects such as facilitation of ophthalmic observation.
The processing performed by the controller 200 (and the data processor 210A), that is, the control of the movement mechanism 70 based on a change in the image parameter, can be on and off (can be activated and inactivated). In other words, the ophthalmic observation apparatus 1 may be configured to turn on and off this control function. For example, the control function can be turned off in the case where automatic movement of the surgical microscope 10 (that is, movement of the surgical microscope 10 at a timing unintended by the user) is desired to be avoided. The switching between activation and inactivation may be conducted through, for example, an operation using the operation device 2 (e.g., foot switch). As another example, the ophthalmic observation apparatus 1 may be configured to switch between activation and inactivation by means of voice recognition technology or other recognition technology. In order to assist the user in such operations and associated processing and works, the ophthalmic observation apparatus 1 may be provided with any of the following kinds of means: a means of informing deterioration in the image parameter (e.g., a decrease in the brightness of a pupil image); a means of informing the start of movement of the surgical microscope 10; a means of asking the user for permission to move the surgical microscope 10, and so forth.
With such a configuration, the automatic movement function of the surgical microscope 10 can be inactivated when, for example, the user does not want to use the automatic movement function or the user is conducting treatment on the subject's eye. In addition, such a configuration allows the ophthalmic observation apparatus 1 to inform that a predetermined condition, such as deterioration in the image parameter, is satisfied. It should be noted that the method of informing may be freely selected or determined. For example, the method of informing may be displaying of information on the display device 3, audio output, turning on of a light source, or other methods. In addition, the condition that triggers the informing (referred to as the third condition) may be freely selected or determine. For example, the third condition may be the same as or different from the first condition and/or the second condition described above.
The ophthalmic observation apparatus 1 may be capable of accepting (receiving) a response from the user to the informing such as informing of deterioration in the image parameter. The response from the user is received by means of a predetermined device such as the operation device 2 or a voice recognition device. Such a device is referred to as a response receiving unit. The controller 200 (and the data processor 210A) may be configured to perform a control of the movement mechanism 70 to move the illumination optical system 30 and the observation optical system 40, based on the response from the user received by the response receiving unit.
With this, the ophthalmic observation apparatus 1 becomes capable of automatically detecting an event that the third condition has been satisfied (e.g., that the image parameter has deteriorated), informing that the third condition is satisfied, receiving a response from the user to this informing, and moving the surgical microscope 10. Therefore, the user can become aware of that the third condition has been satisfied and then issue an instruction to start the search for an optimum position of the illumination optical system 30 and the observation optical system 40.
The data processor 210B shown in
The determining processor 212 is configured to perform determination of whether or not to execute processing performed by the controller 200 (and the data processor 210B). Here, the processing performed by the controller 200 (and the data processor 2108) is the control of the movement mechanism 70 on the basis of a change in the image parameter. For example, the determining processor 212 is configured to perform the determination based on a moving image generated by the surgical microscope 10. In some examples, the determining processor 212 may be configured to determine whether treatment is being performed on the subject's eye based on the moving image generated by the surgical microscope 10, and then to determine not to perform the control of the movement mechanism 70 on the basis of a change in the image parameter if it has been determined that treatment is being performed on the subject's eye. In this way, the ophthalmic observation apparatus 1 of some aspects may be provided with a means configured to analyze an observation image obtained by the surgical microscope 10 and to execute automatic determination of whether or not to move the surgical microscope 10. In some examples, the ophthalmic observation apparatus 1 may be configured to prohibit the movement of the surgical microscope 10 while the user is conducting treatment on the subject's eye. For example, the ophthalmic observation apparatus 1 may be configured to determine, by means of the determining processor 212, whether or not an image of an instrument (e.g., a surgical instrument) exists in the observation image, and to execute a control to prohibit the movement of the surgical microscope 10 if it has been determined that an instrument image is depicted in the observation image. As another example, the ophthalmic observation apparatus 1 may be configured to determine, by means of the determining processor 212, whether or not an image of an instrument depicted in the observation image (moving image) is moving (that is, an instrument image changes with time), and to execute a control to prohibit the movement of the surgical microscope 10 if it has been determined that an instrument image depicted in the observation image has been moving.
With such a configuration, the ophthalmic observation apparatus 1 is capable of carrying out automatic determination of whether or not to perform the automatic search for the position of the optical system of the surgical microscope 10. In addition, such a configuration allows the ophthalmic observation apparatus 1 to prevent the surgical microscope 10 from moving at a timing unintended (unexpected, undesired) by the user, such as while treatment is being performed on the subject's eye.
It is conceivable that the automatic search for the position of the optical system of the surgical microscope 10 cannot yield an image of good quality. In order to address such a case, the ophthalmic observation apparatus 1 may have the function described below (the function executed by the second controller). First, the ophthalmic observation apparatus 1 (e.g., the data processor 210) finds an optimum value of the image parameter, for example, by performing a control of the movement mechanism 70 on the basis of a change in the image parameter. Next, the ophthalmic observation apparatus 1 (e.g., the data processor 210) determines whether or not this optimum value satisfies the fourth condition. When it has been determined that the optimum value of the image parameter satisfies the fourth condition, the ophthalmic observation apparatus 1 (e.g., the controller 200) performs at least one of the following operations: an operation of outputting information (informing); an operation of increasing the amount of the illumination light given by the illumination optical system 30; an operation of increasing the gain of the image sensor of the observation optical system 40; and an operation of changing a determination condition regarding the image parameter. Several specific examples of this function will be described. To begin with, the ophthalmic observation apparatus 1 performs moving image photography while moving the illumination optical system 30 and the observation optical system 40, and also, at the same time (in parallel, simultaneously), performs sequential detection of the brightness value of pupil images, and then obtains a maximum brightness value. This maximum brightness value is an example of the optimum value of the image parameter. The maximum brightness value is the maximum value among brightness values collected by the search for the optimum position of the optical system. Next, the ophthalmic observation apparatus 1 compares the maximum brightness value with the threshold value of the first condition described above, and determines whether the maximum brightness value is less than this threshold value. In the present example, the fourth condition is determined to be satisfied if the maximum brightness value is determined to be less than the threshold value. When the maximum brightness value is determined to be less than the threshold value, the ophthalmic observation apparatus 1 may perform at least one of the operation of informing, the operation of increasing the illumination light amount by the illumination optical system 30, the operation of increasing the gain of the image sensor of the observation optical system 40, and the operation of changing the determination condition regarding the image parameter. The informing operation is performed to inform the user that a pupil image (red reflex image, transillumination image) with a sufficient brightness cannot be obtained by the automatic search for the position of the optical system. Upon receipt of this information, the user realizes the current state and is able to take desired measures. The operation of increasing the illumination light amount and the operation of increasing the gain both contribute to an increase in the brightness of the pupil image (red reflex image, transillumination image). The operation of changing the determination condition regarding the image parameter may be designed, for example, to decrease the threshold value of the first condition. In this case, it can be said that the continuation of medical practice (e.g., surgery) is given priority even if the brightness of the pupil image (red reflex image, transillumination image) is insufficient to a certain extent. The ophthalmic observation apparatus 1 may be configured to select one or more operations from among the informing operation, the operation of increasing the illumination light amount, the operation of increasing the gain, and the operation of changing the determination condition regarding the image parameter, and then execute the operation selected. Furthermore, the ophthalmic observation apparatus 1 may be configured to perform, as a default operation, one or more of the informing operation, the operation of increasing the illumination light amount, the operation of increasing the gain, and the operation of changing the determination condition regarding the image parameter. In addition, the ophthalmic observation apparatus 1 may be configured to perform a combination of two or more of the informing operation, the operation of increasing the illumination light amount, the operation of increasing the gain, and the operation of changing the determination condition regarding the image parameter. In some examples, if brightness equal to or greater than the threshold value of the first condition has not been achieved by the movement of the illumination optical system 30 and the observation optical system 40, the ophthalmic observation apparatus 1 first performs the informing operation. This allows the user to be aware of the fact that the automatic search for the position of the optical system has failed to yield a red reflex image with sufficient brightness. The ophthalmic observation apparatus 1 then increases the amount of light emitted by the light sources 31LA and 31RA, for example, upon receipt of an instruction from the user or automatically. Then, the ophthalmic observation apparatus 1 performs the automatic search for the position of the optical system again. If the brightness of the red reflex image is still insufficient even after the re-execution of the automatic search, the ophthalmic observation apparatus 1 then increases the gains of the image sensors 62L and 62R of the observation optical system 40. Next, the ophthalmic observation apparatus 1 performs the automatic search for the position of the optical system once again. If the brightness of the red reflex image is still insufficient even after this automatic search, the ophthalmic observation apparatus 1 decreases the threshold value of the first condition by a predetermined value. The ophthalmic observation apparatus 1 then performs an automatic search for the position of the optical system one more time. If the brightness of the red reflex image is still insufficient even after this automatic search, the ophthalmic observation apparatus 1 performs the informing operation again. This informing operation is to notify the user that a red reflex image of sufficient brightness cannot be obtained by the series of automatic search processes, and also to propose switching to a manual operation to the user. The manual operation may include, for example, manual adjustment of the position of the illumination optical system 30 and the observation optical system 40, manual adjustment of the illumination light amount, manual adjustment of the gain, and manual adjustment of the threshold value of the first condition.
In this way, the present example enables the ophthalmic observation apparatus 1 to take various kinds of suitable measures in the case where performing the automatic search for the position of the optical system of the surgical microscope 10 does not produce an image of good quality.
When an image of good quality is obtained by the automatic search for the position of the optical system of the surgical microscope 10 (in other words, when the automatic search has been successful), it is also conceivable that an image with the minimum necessary brightness may be obtained. In such a case, the illumination light amount may be decreased in order to reduce the burden on the subject (this operation is performed by the third controller). For example, after performing a successful automatic search for the position of the optical system, the controller 200 of the ophthalmic observation apparatus 1 decreases the illumination light amount by the illumination optical system 30 on condition that the image parameter is included in a predetermined range. The predetermined range is, for example, a range equal to or greater than the threshold value of the first condition.
The present example makes it possible to reduce the burden on the subject by decreasing the illumination light amount on condition that an image of good quality can be obtained, in the event of a successful automatic search for the position of the optical system of the surgical microscope 10.
When a target site to be observed is the posterior eye segment, such as when performing posterior eye segment surgery, the ophthalmic observation apparatus 1 may perform the same or similar processing as or to the above-described case of observing the anterior eye segment. For example, the ophthalmic observation apparatus 1 may be configured to perform the following processes: a process of performing movement of the illumination optical system 30 and the observation optical system 40 by the movement mechanism 70 while generating a moving image by the surgical microscope 10; a process of sequentially analyzing, by the analyzing processor 211, a plurality of still images included in the moving image to sequentially detect images of a predetermined site of the posterior eye segment (e.g., optic nerve head); and a process of controlling the movement mechanism 70 based on a change in a predetermined image parameter (e.g., brightness) of the images sequentially detected. With this configuration, the same or similar actions and the same or similar advantageous effects as or to the case of anterior eye segment observation can also be achieved in the case of posterior eye segment observation. Any of the matters and items described for the case of anterior eye segment observation can be combined with the case of posterior eye segment observation.
The operation and the usage mode of the ophthalmic observation apparatus 1 will be described.
(S1: Commence Generation and Display of Live Image with Coaxial Illumination)
To begin with, the user performs a predetermined operation using the operation device 2 to select the coaxial illumination mode and cause the ophthalmic observation apparatus 1 to start generating and displaying a live image of the subject's eye (the anterior eye segment thereof). More specifically, the surgical microscope 10 illuminates the subject's eye by the illumination optical system 30 (the first illumination optical systems 31L and 31R), and at the same time (in parallel, simultaneously) generates digital image data (moving image, video) of the subject's eye by the image sensors 62L and 62R. The generated moving image (the live image 301) is displayed in real time on the display device 3 (see
(S2: Detect Pupil Center from Live Image)
Next, the data processor 210 (e.g., the analyzing processor 211) detects the pupil center from the live image (frames thereof) whose generation has started in the step S1. In some aspects, the data processor 210 first applies image processing such as binarization and/or segmentation to detect the corneal ring 302 (see
(S3: Align the Optical Axis of Optical System with Pupil Center)
Next, the ophthalmic observation apparatus 1 aligns the optical axis of the surgical microscope 10 (the illumination optical system 30, the observation optical system 40) with the pupil center 304 identified in the step S2. For example, the ophthalmic observation apparatus 1 performs the following processes: registration (position matching, position adjustment) between a newly acquired frame and the frame used to detect the pupil center 304; identification of the position corresponding to the pupil center in the new frame based on the result of the registration; and movement of the illumination optical system 30 and the observation optical system 40 in such a manner that the optical axis of the surgical microscope is disposed at the position corresponding to the pupil center.
In some examples, the user can manually place the optical axis of the surgical microscope 10 at the pupil center.
The ophthalmic observation apparatus 1 (the analyzing processor 211) detects a pupil image from the live image after the optical axis of the surgical microscope 10 has been placed at the pupil center 304 by the step S3.
Next, the ophthalmic observation apparatus 1 calculates the brightness of the pupil image detected in the step S4. As the first example of the calculation of the brightness of the pupil image, the data processor 210 (e.g., the analyzing processor 211) performs the following processes: a process of dividing the pupil image (and its vicinity region) into a plurality of subregions; a process of determining a representative value of the brightness of each subregion (the representative value may be average, maximum, mode, median, or any other statistical value); and a process of determining the brightness of the pupil image based on the distribution of the representative values of brightness obtained for the plurality of subregions. The process of determining the brightness of the pupil image from the distribution may include any statistical processing, such as calculation of the average, identification of the maximum, identification of the mode, or identification of the median. As the second example of the calculation of the brightness of the pupil image, the data processor 210 (e.g., the analyzing processor 211) may determine the brightness of the pupil image based on the brightness values of a plurality of pixels (all the pixels or a selected group of pixels) of the pupil image using a predetermined statistical technique.
(S6: Is Brightness Equal to or Larger than Threshold Value and Maximum?)
Next, the ophthalmic observation apparatus 1 (the controller 200 or the data processor 210) executes both determination of whether the brightness of the pupil image calculated in the step S5 is equal to or larger than a predetermined threshold value, and determination of whether the brightness of the pupil image is the maximum (local maximum). It should be noted that the ophthalmic observation apparatus 1 may be configured to perform only one of the two determination processes. The threshold value determination (thresholding) is performed by comparing the brightness of the pupil image with the threshold value. This threshold value may be, for example, the threshold value of the first condition described above. The maximum determination is performed by monitoring the change in brightness caused by the movement of the illumination optical system 30 and the observation optical system 40. For example, by searching for a position at which the change in brightness is switched from increasing to decreasing, it can be determined whether a certain position of the surgical microscope 10 (the illumination optical system 30 and the observation optical system 40) is the position corresponding to maximum brightness (that is, the positions of the illumination optical system 30 and the observation optical system 40 where the brightness becomes the maximum can be found).
If it is determined that the brightness of the pupil image calculated in the step S5 is equal to or larger than the threshold value and also that the brightness of the pupil image is the maximum (S6: Yes), the operation proceeds to the step S8. If it is determined that the brightness of the pupil image calculated in the step S5 is less than the threshold value and/or that the brightness of the pupil image is not the maximum (S6: No), the operation proceeds to the step S7.
If the determination result “No” is issued in the step S6, the ophthalmic observation apparatus 1 (the controller 200, the movement mechanism 70) moves the illumination optical system 30 and the observation optical system 40. The movement direction may be freely selected or determined, and may be any one or more of the vertical direction (the Z direction), the horizontal direction (the X direction), and the front and back direction (the Y direction). Also, as described above, the movement direction may be determined based on the change in the brightness. The movement amount (movement distance) may be determined in advance, or may be determined based on the change in the brightness.
If the determination result “Yes” is issued in the step S6, the ophthalmic observation apparatus 1 (the controller 200) ends the movement of the illumination optical system 30 and the observation optical system 40 (End). According to the present example of the operation and the usage mode, the illumination optical system 30 and the observation optical system 40 can be moved to and disposed at a position where the brightness of the pupil image not only becomes equal to or larger than the threshold value but also becomes maximized. In other words, the present example makes it possible to obtain a red reflex image that has the brightness equal to or larger than the threshold value and that is of the maximum brightness.
Any of the matters and items described above can be combined with the operation and the usage mode of the present example. For example, when the brightness of the pupil image falls below a predetermined value (the threshold value of the second condition described above), the ophthalmic observation apparatus 1 can move the illumination optical system 30 and the observation optical system 40 again to search for a position where the brightness of the pupil image is maximized, and then move and place the illumination optical system 30 and the observation optical system 40 to and at the searched position. Furthermore, if the brightness equal to or larger than the threshold value cannot be achieved even after the movement of the illumination optical system 30 and the observation optical system 40, the ophthalmic observation apparatus 1 may execute processing such as the operation of informing, the operation of increasing the illumination light amount, the operation of increasing the gain of the image sensor, or the operation of reducing the threshold value. In addition, after the illumination optical system 30 and the observation optical system 40 are moved to a position where the brightness of the pupil image is equal to or larger than the threshold value and is the maximum, the illumination light amount may be decreased within the range where the brightness of the pupil image is equal to or larger than the predetermined value, thereby reducing the burden on the subject.
Some embodiment examples (e.g., the ophthalmic observation apparatus 1 described above) provide a method of controlling an ophthalmic observation apparatus. It is possible to combine any items or matters relating to the ophthalmic observation apparatus 1 of the above embodiment examples with the example of the method described below.
An ophthalmic observation apparatus controlled by the method of an aspect example includes an optical system for observing a subject's eye (e.g., the illumination optical system 30 and the observation optical system 40 moved by the movement mechanism 70) and a processor (e.g., the controller 200 and the data processor 210). The method of the present aspect example first causes the ophthalmic observation apparatus to perform movement of the optical system and generation of a moving image of the subject's eye by the optical system in parallel. Further, the method of the present aspect example causes the processor to sequentially analyze a plurality of still images included in the moving image to sequentially detect images of a predetermined site of the subject's eye. In addition, the method of the present aspect example causes the processor to perform movement of the optical system based on a change in a predetermined image parameter of the images of the predetermined site sequentially detected.
The method of the present aspect example is capable of achieving the same actions and effects as those of the ophthalmic observation apparatus 1 of the above embodiment examples. In addition, by combining any of the matters and items relating to the ophthalmic observation apparatus 1 of the above embodiment examples with the method of the present aspect example, the resulting method becomes capable of achieving the actions and effects corresponding to the combined matters and/or items.
Some embodiment examples provide a program causing a computer to execute the method of the aspect example described above. It is possible to combine any of the matters and items relating to the ophthalmic observation apparatus 1 of the above embodiment examples with such a program.
The program thus configured is capable of achieving the same actions and effects as those of the ophthalmic observation apparatus 1 of the above embodiment examples. In addition, by combining any of the matters and items relating to the ophthalmic observation apparatus 1 of the above embodiment examples with the program, the resulting program is capable of achieving the actions and effects corresponding to the combined matters and/or items.
Some embodiment examples provide a computer-readable non-transitory recording medium storing the program described above. It is possible to combine any of the matters and items relating to the ophthalmic observation apparatus 1 of the above embodiment examples with such a recording medium. The non-transitory recording medium may be in any form, and examples thereof include magnetic disks, optical disks, magneto-optical disks, and semiconductor memories.
The recording medium thus configured is capable of achieving the same actions and effects as those of the ophthalmic observation apparatus 1 of the above embodiment examples. In addition, by combining any of the matters and items relating to the ophthalmic observation apparatus 1 of the above embodiment examples with the recording medium, the resulting recording medium is capable of achieving the actions and effects corresponding to the combined matters and/or items.
The present disclosure can perform automatic optimization of observation environment by automatically determining the position where return intensity of fundus reflection of coaxial illumination is appropriate (equal to or larger than a threshold value, maximized) and moving the microscope optical system, for example. In addition, the present disclosure can maintain good observation environment by performing the same or a similar operation when the subject or the subject's eye moves. Furthermore, the same or a similar operation may be performed in the cases of employing an illumination method other than coaxial illumination (e.g., oblique illumination). For example, if a user's preference is taken into consideration, preset information may be prepared for each user.
In addition, the present disclosure allows the user to always observe the eye under optimal illumination conditions, as long as the operation according to the present disclosure is functioning properly. As a result of this, the success rate of surgery can be improved, the operation time can be shortened, and the amount of illumination can be reduced. These effects reduce the burden on the patient. For example, by employing the present disclosure in cataract surgery, an appropriate red reflex image can be obtained with coaxial illumination, which makes it possible to achieve appropriate and efficient works such as phacoemulsification (crystalline lens emulsification and aspiration) works and posterior capsule polishing works.
The embodiments described in the present disclosure are merely examples, and any modification, omission, addition, substitution, etc. can be made within the scope of the present disclosure and its equivalents.
The present application claims priority to U.S. Provisional Patent Application Ser. No. 63/106,087, filed Oct. 27, 2020, entitled “APPARATUS AND METHOD FOR OPHTHALMIC OBSERVATION”, the entirety of which is incorporated herein by reference.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2020/045760 | 12/9/2020 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 63106087 | Oct 2020 | US |
