This disclosure relates generally to retinal imaging technologies, and in particular but not exclusively, relates to illumination techniques for retinal imaging.
Retinal imaging is a part of basic eye exams for screening, field diagnosis, and progress monitoring of many retinal diseases. A high fidelity retinal image is important for accurate screening, diagnosis, and monitoring. Bright illumination of the posterior interior surface of the eye (i.e., retina) through the pupil improves image fidelity but often creates optical aberrations or image artifacts, such as corneal reflections, iris reflections, or lens flare, if the retinal camera and illumination source are not adequately aligned with the eye. Simply increasing the brightness of the illumination does not overcome these problems, but rather makes the optical artifacts more pronounced, which undermines the goal of improving image fidelity.
Accordingly, camera alignment is very important, particularly with conventional retinal cameras, which typically have a very limited eyebox due to the need to block the deleterious image artifacts listed above. The eyebox for a retinal camera is a three dimensional region in space typically defined relative to an eyepiece of the retinal camera and within which the center of a pupil or cornea of the eye should reside to acquire an acceptable image of the retina. The small size of conventional eyeboxes makes retinal camera alignment difficult and patient interactions during the alignment process often strained.
Various solutions have been proposed to alleviate the alignment problem. For example, moving/motorized stages that automatically adjust the retina-camera alignment have been proposed. However, these stages tend to be mechanically complex and substantially drive up the cost of a retinal imaging platform. An effective and low cost solution for efficiently and easily achieving eyebox alignment of a retinal camera would improve the operation of retinal cameras.
Non-limiting and non-exhaustive embodiments of the invention are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified. Not all instances of an element are necessarily labeled so as not to clutter the drawings where appropriate. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles being described.
Embodiments of a system, apparatus, and method of operation of a retinal camera with a dynamic illuminator having an expanded eyebox are described herein. In the following description numerous specific details are set forth to provide a thorough understanding of the embodiments. One skilled in the relevant art will recognize, however, that the techniques described herein can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring certain aspects.
Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases “in one embodiment” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
High fidelity retinal images are important for screening, diagnosing, and monitoring many retinal diseases. To this end, reducing or eliminating instances of image artifacts that occlude, or otherwise malign portions of the retinal image is desirable.
Conventional imaging systems have relatively small eyeboxes, which require precise alignment to avoid image artifacts from entering the image path. Embodiments described herein provide a dynamic illuminator that changes its illumination pattern based upon a detected alignment between the retinal imaging system and an eye. These dynamic changes in the illumination pattern expand the eyebox without use of complicated or costly mechanical components. The expanded eyebox eases the alignment burden while reducing the instances of image artifacts occluding or otherwise maligning the captured retinal image. The dynamic illuminator combines two different illumination architectures—one when the eye is roughly aligned with the optical axis or gaze direction of the eye (referred to herein as a circular illumination pattern) and one when the eye is offset from the optical axis or gaze direction of the eye (referred to herein as a non-circular illumination pattern or stacked illumination). By dynamically switching between these two illumination architectures, the eyebox of the retinal imaging system described herein may be expanded by 2× or more over conventional ring illuminators.
The optical relay system serves to direct (e.g., pass or reflect) illumination light 280 output from dynamic illuminator 205 along an illumination path through the pupil of eye 270 to illuminate retina 275 while also directing image light 285 of retina 275 (i.e., the retinal image) along an image path to image sensor 210. Image light 285 is formed by the scattered reflection of illumination light 280 off of retina 275. In the illustrated embodiment, the optical relay system further includes beam splitter 250, which passes at least a portion of image light 285 to image sensor 210 while also directing display light 290 output from display 225 to eye 270. Beam splitter 250 may be implemented as a polarized beam splitter, a non-polarized beam splitter (e.g., 90% transmissive and 10% reflective, 50/50 beam splitter, etc.), a dichroic beam splitter, or otherwise. The optical relay system includes a number of lenses, such as lenses 235, 240, and 245, to focus the various light paths as needed. For example, lens 235 may include one or more lensing elements that collectively form an eyepiece that is displaced from the cornea of eye 270 by an eye relief 295 during operation. Lens 240 may include one or more lens elements for bring image light 285 to a focus on image sensor 210. Lens 245 may include one or more lens elements for focusing display light 290. It should be appreciated that optical relay system may be implemented with a number and variety of optical elements (e.g., lenses, reflective surfaces, diffractive surfaces, etc.).
In one embodiment, display light 290 output from display 225 is a fixation target or other visual stimuli. The fixation target not only can aid with obtaining alignment between retinal imaging system 200 and eye 270 by providing visual feedback to the patient, but may also give the patient a fixation target upon which the patient can accommodate their vision. Display 225 may be implemented with a variety of technologies including an liquid crystal display (LCD), light emitting diodes (LEDs), various illuminated shapes (e.g., an illuminated cross or concentric circles), or otherwise.
Controller 215 is coupled to image sensor 210, display 225, dynamic illuminator 205, and alignment tracker 230 to choreograph their operation. Controller 215 may include software/firmware logic executing on a microcontroller, hardware logic (e.g., application specific integrated circuit, field programmable gate array, etc.), or a combination of software and hardware logic. Although
Image sensor 210 may be implemented using a variety of imaging technologies, such as complementary metal-oxide-semiconductor (CMOS) image sensors, charged-coupled device (CCD) image sensors, or otherwise. In one embodiment, image sensor 210 includes an onboard memory buffer or attached memory to store retinal images.
Alignment tracker 230 operates to track alignment between retinal imaging system 200 and eye 270. Alignment tracker 230 may operate using a variety of different techniques to track the relative positions of eye 270 and retinal imaging system 200 including pupil tracking, retina tracking, iris tracking, or otherwise. In one embodiment, alignment tracker 230 includes one or more infrared (IR) emitters to track eye 270 via IR light while retinal images are acquired with visible spectrum light. In such an embodiment, IR filters may be positioned within the image path to filter the IR tracking light. In other embodiments, the tracking illumination is temporally offset from image acquisition.
During operation, controller 115 operates dynamic illuminator 105 and retinal camera 110 to capture one or more retinal images. Dynamic illuminator 105 is dynamic in that its illumination pattern is not static; but rather, is dynamically changed under the influence of controller 215 based upon the determined alignment with eye 270 (discussed in detail below). Illumination light 280 is directed through the pupil of eye 270 to illuminate retina 275. The scattered reflections from retina 275 are directed back along the image path through an aperture in center baffle 255 to image sensor 210. Center baffle 255 operates to block deleterious reflections and light scattering that would otherwise malign the retinal image while passing the image light itself. The illumination patterns output by dynamic illuminator 205 are selected based upon the current alignment to reduce deleterious image artifacts. Image artifacts may arise from light scattering by the human lens within eye 270, reflections from the cornea/iris, or even direct specular reflections of illumination light 280 from retina 275. Direct specular reflections from retina 275 or the cornea/iris can create washed out regions (e.g., image artifacts 105) in the retinal image. The dynamic changes in the illumination patterns output from dynamic illuminator 205 serve to direct these specular reflections off axis from the image path and therefore blocked by the field stop or center baffle 255.
As mentioned, dynamic illuminator 300 includes arrays of illumination sources 325 that extend out from center baffle 305 to provide the source of light for illuminating retina 275. In the illustrated embodiment, illumination arrays 315 and 320 extend along substantially orthogonal linear axes 306 and 307, respectively, forming a sort of plus sign or cross-like shape. In the illustrated embodiment, linear axes 306 and 307 are radial lines that pass substantially through the center of aperture 310. In one embodiment, illumination sources 325 within illumination array 315 are symmetrically positioned about linear axis 307 while illumination sources 325 within illumination array 320 are symmetrically positioned about linear axis 306. Each illumination array includes two sections that extend from opposing sides of center baffle 305 and aperture 310. Illumination arrays 315 and 320 include discrete locations of independently controlled illumination. In other words, illumination sources 325 can be independently enabled or disabled under the influence of controller 215 to generate distinct illumination patterns. In one embodiment, illumination sources 325 are implemented as distinct LED sources. In other embodiments, illumination sources 325 may be implemented with a variety of technologies and configurations capable of providing distinct locations of independently controllable illumination light. For example, each section of the illumination arrays 315 and 320 may share a common backlight, but have controllable masks (e.g., LCD screens) to selectively filter and control the location of light illumination. Other illumination technologies may be used. Furthermore, although each illumination array 315 and 320 is illustrated as including eight illumination sources 325, implementations may include more or less illumination sources 325. In one embodiment, illumination sources 325 have the following separation pitches and sizes: L1=6.5 mm, L2=10 mm, L3=13 mm, L4=16 mm, L5=3 mm, and L6=2 mm. Of course, other sizes and separation pitches may be implemented.
In the illustrated embodiment, center baffle 305 has a cone shape that surrounds and extends out towards the eyepiece lens 235 from aperture 310. The sides of center baffle 305 overlap a portion of the inner most illumination sources 325 that are immediately adjacent to center baffle 305. This partial overlap causes center baffle 305 to partially block or cast a shadow when backlit by the innermost illumination sources 325, but does not block the other illumination sources 325. The shadow serves to substantially separate and isolate the image path from the illumination path thus reducing the crosstalk between these paths and reducing image artifacts in the retinal images. When the innermost illumination sources 325 are illuminated, center baffle 305 blocks illumination ray angles that create poor image quality due to scattering in the eye lens and reflections from the cornea.
Deleterious image artifacts are further isolated and reduced by the use of illumination baffles 330 surrounding each illumination source 325. Illumination baffles 330 serve to constrain the emission divergence pattern of illumination sources 325, and in some embodiments also constrain the effective die size of illumination sources 325 by covering over portions of each illumination source 325. Illumination baffles 330 also reduce the dependence of the illumination path on fabrication deviances/tolerances between batches or instances of the discrete illumination sources 325 particularly since many LED sources, or other types of illumination sources, do not generate a precise illumination pattern or collimated light. Illumination baffles 330 may be implemented as discrete baffles or part of an integrated shroud or molding assembly. The molding assembly may include a discrete molding for each half of a given illumination array 315 or 320, or alternatively, illumination baffles 330 along with center baffle 305 may be fabricated from a single contiguous assembly. In the illustrated embodiment, center baffle 305 has a circular cross-sectional shape (about a center optical axis of the image path passing through aperture 310) while illumination baffles 330 have a rectangular cross-sectional shape. Of course, other cross-sectional shapes may be used to fine tune the illumination and image paths. For example, illumination baffles 330 may also have a circular cross-sectional shape. In one embodiment, center baffle 305 and illumination baffles 330 have the following dimensions: D1=13 mm, D2=10.60 mm, L5=3 mm, L6=2 mm, L7=1.5 mm, L8=5.5 mm. Of course, other dimensions may be implemented.
In a process block 405, the retinal imaging process is initiated. Initiation may include the user selecting a power button from user interface 220. In a process block 410, alignment tracker 230 commences tracking and determining the alignment between retinal camera system 200 and eye 270. In particular, tracking may be determined as a relative measurement between eyepiece lens 235 and the pupil, iris, or retina of eye 270. A variety of different alignment tracking techniques may be implemented including pupil tracking, iris tracking, retinal tracking, trial and error, etc. The alignment tracking is used to determine, which of at least two illumination schemes should be used for illuminating retina 275 during image acquisition. The transition between these illumination schemes may be abrupt or a smooth fading there between as the relative alignment wanders between a central alignment and an offset alignment.
In decision block 415, if retinal camera system 200 is determined to be centrally aligned with the gaze direction 271 (e.g., optical axis of eye 270) within a defined threshold, then process 400 continues to a process block 420. In process block 420, dynamic illuminator 205 is operated by controller 215 to generate a circular illumination pattern for illuminating retina 275 through the pupil of eye 270.
The emission divergence patterns of illumination sources 501 are constrained and controlled by illumination baffles 330 and a shadow cast by center baffle 305 (process block 425). Center baffle 305 serves to block the portion of the illumination light output from inner illuminator sources 501 that would cause deleterious scattering in eye 270. As further illustrated in
In process block 430, the retinal image passes through aperture 310 where center baffle 305 further blocks deleterious reflections and other stray refractions (process block 435) before image sensor 210 captures image light 285 forming the retinal image.
Returning to decision block 415, if retinal camera system 200 is determined to be offset from the gaze direction 271 by a defined threshold, then process 400 continues to process blocks 445 to 455. In process blocks 445 to 455, dynamic illuminator 205 is operated by controller 215 to generate a non-circular illumination pattern (also referred to as a stacked illumination pattern) for illuminating retina 275 through the pupil of eye 270.
Illumination arrays 915, 920, and 922 operate in a similar manner as discussed above in connection with illumination arrays 315 and 320, except that the option additional diagonal illumination array 922 provides additional illumination flexibility. The illustrated embodiment of each illumination array 915, 920, and 922 includes ten discrete illumination sources 325 compared to eight discrete illumination sources 325 for each illumination array 315 and 320. While the number of discrete illumination sources 325 per illumination array may be adjusted, the two additional discrete illumination sources 325 provide finer granular control over the illumination patterns.
Center baffle 905 also operates in a similar functional manner as center baffle 305; however, uses two cylindrical shroud walls 910 and 912 to precisely confine the emission divergence pattern (and strategically cast shadows onto eye 270) as opposed to the single cone shaped center baffle 305. The straight, cylindrical shape of shroud walls 910 and 912 may be simpler to manufacture versus the angled cone shape of center baffle 305. As illustrated, outer cylindrical shroud wall 912 surrounds the inner cylindrical shroud wall 910. Both cylindrical shroud walls 910 and 912 extend from the plane of aperture 310; however, inner cylindrical shroud wall 910 extends to a greater height than outer cylindrical shroud wall 912. Furthermore, the inner most discrete illumination sources 325 of each illumination array 915, 920, and 922 (i.e., the two inner most discrete illumination sources from each illumination array disposed on either side of, and immediately adjacent to, aperture 310) are disposed radially between inner cylindrical shroud wall 910 and outer cylindrical shroud wall 912. The remaining discrete illumination sources 325 are all disposed radially outside of cylindrical shroud wall 912. This intermediate position of inner discrete illumination sources 325 enables cylindrical shroud walls 910 and 912 to confine the inner and outer edges (i.e., emission divergence pattern) of the illumination path output from the inner discrete illumination sources 325. Correspondingly, the heights of cylindrical shroud walls 910 and 912 are also selected to control the inner edge of the illumination paths output from the second inner ring of discrete illumination sources 325 (i.e., discrete illumination sources 325 radially outside of, but immediately adjacent to, cylindrical shroud wall 912).
The processes explained above are described in terms of computer software and hardware. The techniques described may constitute machine-executable instructions embodied within a tangible or non-transitory machine (e.g., computer) readable storage medium, that when executed by a machine will cause the machine to perform the operations described. Additionally, the processes may be embodied within hardware, such as an application specific integrated circuit (“ASIC”) or otherwise.
A tangible machine-readable storage medium includes any mechanism that provides (i.e., stores) information in a non-transitory form accessible by a machine (e.g., a computer, network device, personal digital assistant, manufacturing tool, any device with a set of one or more processors, etc.). For example, a machine-readable storage medium includes recordable/non-recordable media (e.g., read only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).
The above description of illustrated embodiments of the invention, including what is described in the Abstract, is not intended to be exhaustive or to limit the invention to the precise forms disclosed. While specific embodiments of, and examples for, the invention are described herein for illustrative purposes, various modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize.
These modifications can be made to the invention in light of the above detailed description. The terms used in the following claims should not be construed to limit the invention to the specific embodiments disclosed in the specification. Rather, the scope of the invention is to be determined entirely by the following claims, which are to be construed in accordance with established doctrines of claim interpretation.
This application is a continuation of U.S. Ser. No. 16/457,191 filed Jun. 28, 2019, which claims the benefit of U.S. Application No. 62/698,457, filed on Jul. 16, 2018, the contents both of which are incorporated herein by reference.
Number | Name | Date | Kind |
---|---|---|---|
5585873 | Shalon et al. | Dec 1996 | A |
7677730 | Shimizu | Mar 2010 | B2 |
8708490 | Baranton et al. | Apr 2014 | B2 |
9743832 | de Paz Sicam et al. | Aug 2017 | B2 |
20020089644 | Goldfain | Jul 2002 | A1 |
20040196432 | Su | Oct 2004 | A1 |
20050134796 | Zelvin | Jun 2005 | A1 |
20130010260 | Tumlinson et al. | Jan 2013 | A1 |
20130057828 | de Smet | Mar 2013 | A1 |
20160029887 | Su | Feb 2016 | A1 |
20170164830 | Huang et al. | Jun 2017 | A1 |
20180070819 | Kanamori et al. | Mar 2018 | A1 |
20180220888 | Tumlinson | Aug 2018 | A1 |
20190046031 | Kramer | Feb 2019 | A1 |
Number | Date | Country |
---|---|---|
201651318 | Nov 2010 | CN |
201851036 | Jun 2011 | CN |
51150897 | Dec 1976 | JP |
9808439 | Mar 1998 | WO |
2003039332 | May 2003 | WO |
2012018991 | Feb 2012 | WO |
WO-2012018991 | Feb 2012 | WO |
2014127134 | Aug 2014 | WO |
2017025583 | Feb 2017 | WO |
Entry |
---|
Japanese Office Action, dated Jun. 27, 2022, in corresponding Japanese Patent Application No. 2020-571814, 2 pages. |
DeHoog, E., and J. Schwiegerling, “Optimal Parameters for Retinal Illumination and Imaging in Fundus Cameras,” Applied Optics 47(36):6769-6777, Dec. 2008. |
International Search Report and Written Opinion, dated Oct. 17, 2019, for corresponding International Patent Application No. PCT/US2019/040606, 7 pages. |
Non Final Office Action dated Apr. 7, 2021 in U.S. Appl. No. 16/456,191, filed Jun. 28, 2019, 12 pages. |
Japanese Office Action, dated Feb. 14, 2022, in corresponding Japanese Patent Application No. 2020-571814 3 pages. |
Supplementary European Search Report, dated Mar. 14, 2022, in corresponding European Patent Application No. 19837128.8-1126, 5 pages. |
Number | Date | Country | |
---|---|---|---|
20220117487 A1 | Apr 2022 | US |
Number | Date | Country | |
---|---|---|---|
62698457 | Jul 2018 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 16457191 | Jun 2019 | US |
Child | 17526629 | US |