Patent Application
20240206727
Geographic atrophy (GA) is the late stage of nonexudative (dry) age-related macular degeneration (AMD), which is a major cause of vision loss worldwide. Geographic atrophy is characterized by the loss of photoreceptors, retinal pigment epithelium (RPE), and choriocapillaris (CC), and leads to irreversible vision loss where the geographic atrophy is present. Geographic atrophy is also known as complete RPE and outer retinal atrophy (cRORA). Currently there are no Food and Drug Administration approved treatments to prevent the formation or progression of geographic atrophy, but several promising therapeutic treatment clinical trials using complement inhibitors are underway.
Rather than using visual acuity as a clinical trial endpoint, most studies use the slowing of the GA enlargement rate (ER) as the clinical trial endpoint because vision is usually affected late in the disease process when the GA progresses into the foveal region. There has been a great deal of interest in identifying GA that is more likely to enlarge more rapidly, hoping not only to understand the underlying disease pathophysiology responsible for GA growth, but also to help facilitate the testing of promising therapies to slow the progression of GA against more rapidly growing GA so that clinical trials can be of shorter duration.
An automated and accurate approach to identify, segment, and quantify GA would be of great interest and importance for following patients in clinical practice and confirming the effectiveness of treatments in clinical trials, as would automated and accurate techniques for predicting GA enlargement rates.
This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
In some embodiments, a computer-implemented method of automatically predicting progression of age-related macular degeneration is provided. An image analysis computing system receives optical coherence tomography data (OCT data). The image analysis computing system determines an optical attenuation coefficient for each pixel of the OCT data to create optical attenuation coefficient data (OAC data) corresponding to the OCT data. The image analysis computing system determines an area exhibiting geographic atrophy based on at least one of the OCT data and the OAC data. The image analysis computing system measures one or more attributes within an adjacent area that is adjacent to the area exhibiting geographic atrophy, and the image analysis computing system determines a predicted enlargement rate based on the one or more attributes within the adjacent area.
In some embodiments, a computer-implemented method of automatically detecting an area of an eye exhibiting geographic atrophy is provided. An image analysis computing system receives optical coherence tomography data (OCT data). The image analysis computing system determines an optical attenuation coefficient for each pixel of the OCT data to create optical attenuation coefficient data (OAC data) corresponding to the OCT data, and the image analysis computing system determines an area exhibiting geographic atrophy based on the OAC data.
In some embodiments, computer-readable media having computer-executable instructions stored thereon are provided. The instructions, in response to execution by an image analysis computing system, cause the image analysis computing system to perform one of the methods described above. In some embodiments, an image analysis computing system configured to perform one of the methods described above is provided.
The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same become better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein:
Traditionally, geographic atrophy has been imaged with its enlargement rate measured using 3 major approaches: color fundus imaging (CFI), fundus autofluorescence (FAF), and optical coherence tomography (OCT). Although CFI is of historical interest, FAF and OCT imaging are currently used in clinical practice and clinical research because these imaging modalities provide better contrast for detecting the loss of the RPE, which is the sine qua non of GA.
Whereas FAF imaging provides only a 2-dimensional view of the fundus without any depth information, OCT imaging, including both spectral domain OCT (SD-OCT) and swept-source OCT (SS-OCT), are useful to visualize GA, quantify GA and measure the growth of GA. The depth-resolved nature of OCT imaging allows for layer specific visualization and the ability to differentiate the extent of anatomical changes across different layers.
In addition to using OCT B-scans, en face OCT imaging is a useful strategy for visualizing GA, and the use of boundary-specific segmentation by using a choroidal slab under the RPE allows for an en face image that specifically accentuates the choroidal hypertransmission defects (hyperTDs) that arise when the RPE is absent.
In the present disclosure, a novel deep learning approach is provided to identify and segment GA areas using optical attenuation coefficients (OACs) calculated from OCT data. Novel en face OAC images are used to identify and visualize GA, and machine learning models are used for the task of automatic GA identification and segmentation. In some embodiments, once GA areas are segmented, measurements in an adjacent area to the GA are obtained of at least one of an RPE-BM distance, an outer retinal thickness, and a choriocapillaris flow deficit, and a predicted enlargement rate of the GA is determined based on the measurements.
According to Classification of Atrophy Meetings (CAM) consensus, the definition of geographic atrophy or complete retinal pigment epithelial and outer retinal atrophy (cRORA) is defined by 3 inclusive OCT criteria: (1) region of hyperTD with at least 250 μm in its greatest linear dimension, (2) zone of attenuation or disruption of the RPE of at least 250 μm in its greatest linear dimension, and (3) evidence of overlying photoreceptor degeneration; and 1 exclusive criteria: the presence of scrolled RPE or other signs of an RPE tear. This definition of geographic atrophy or cRORA relies solely on average B-scans, but en face imaging of geographic atrophy using the subRPE slab is a convenient alternative for the detection of geographic atrophy using fundus autofluorescence and conventional OCT B-scans. The proposed approaches described herein using OAC data are particularly suitable for geographic atrophy identification because they allow en face views with direct three-dimensional information of RPE attenuation and disruption. OAC quantifies the tissues' ability to attenuate (absorb and scatter) light, meaning that it is particularly useful to identify high pigmentation (or the lack of) in retinal tissues.
Using a custom slab and en face imaging strategy with OAC data, the RPE may be visualized with strong contrast. When RPE cells die and lose pigments, their OAC values are reduced as well, resulting in a dark appearance on the false color images described below. In addition to the enhanced contrast for attenuated or disrupted RPE, the OAC approach described herein also provides similar depth-resolved advantages available in traditional OCT approaches. By incorporating three different en face images from the same slab in the false color images based on the OAC data, depth-resolved information—namely the RPE elevation information—is provided on an en faceview. This approach is also useful for identifying drusen or other forms of RPE elevation in AMD eyes.
The illustration shows a layer of rods and cones 102 (photoreceptors), a retinal pigment epithelium 104 (also referred to as the RPE), a Bruch's membrane 106 (also referred to as the BM), and a choriocapillaris 118. The Bruch's membrane 106 includes an RPE basement membrane 108, an inner collagenous zone 110, a region of central elastic fiber bands 112, an outer collagenous zone 114, and a choroid basement membrane 116. Those of ordinary skill in the art will understand the location and biological function of the labeled structures of the diagram 100, as well as the anatomy of portions of the eye that are not illustrated.
As shown, the system 200 includes an image analysis computing system 202 and an optical coherence tomagraphy (OCT) imaging system 204. The OCT imaging system 204 is configured to obtain OCT data representing an eye of a subject 206, and to provide the OCT data to the image analysis computing system 202 for segmentation, measurement, and prediction.
In some embodiments, the OCT imaging system 204 is configured to use light waves to generate both en face imagery (also referred to as A-lines) at one or more depths and cross-sectional imagery (also referred to as B-lines) at one or more locations. In some embodiments, the OCT imaging system 204 may use swept-source OCT (SS-OCT) technology. In some embodiments, the OCT imaging system 204 may use spectral-domain OCT (SD-OCT) technology. In some embodiments, other forms of OCT technology may be used. One non-limiting example of an OCT imaging system 204 suitable for use with the present disclosure is the PLEX® Elite 9000, manufactured by Carl Zeiss Meditec of Dublin, CA. This instrument uses a 100 kHz light source with a 1050 nm central wavelength and a 100 nm bandwidth, resulting in an axial resolution of about 5.5 μm and a lateral resolution of about 20 μm estimated at the retinal surface. Such an instrument may be used to create 6×6 mm scans, for which there are 1536 pixels on each A-line (3 mm), 600 A-lines on each B-scan, and 500 sets of twice-repeated B-scans.
In some embodiments, the OCT imaging system 204 is communicatively coupled to the image analysis computing system 202 using any suitable communication technology, including but not limited to wired technologies (e.g., Ethernet, USB, FireWire, etc.), wireless technologies (e.g., WiFi, WiMAX, 3G, 4G, LTE, Bluetooth, etc.), exchange of removable computer-readable media (e.g., flash memory, optical disks, magnetic disks, etc.), and combinations thereof. In some embodiments, the OCT imaging system 204 performs some processing of the OCT data before providing the OCT data to the image analysis computing system 202 and/or upon request by the image analysis computing system 202.
As shown, the image analysis computing system 202 includes one or more processors 302, one or more communication interfaces 304, an image data store 308, a model data store 320, and a computer-readable medium 306.
In some embodiments, the processors 302 may include any suitable type of general-purpose computer processor. In some embodiments, the processors 302 may include one or more special-purpose computer processors or AI accelerators optimized for specific computing tasks, including but not limited to graphical processing units (GPUs), vision processing units (VPTs), and tensor processing units (TPUs).
In some embodiments, the communication interfaces 304 include one or more hardware and or software interfaces suitable for providing communication links between components. The communication interfaces 304 may support one or more wired communication technologies (including but not limited to Ethernet, FireWire, and USB), one or more wireless communication technologies (including but not limited to Wi-Fi, WiMAX, Bluetooth, 2G, 3G, 4G, 5G, and LTE), and/or combinations thereof.
As shown, the computer-readable medium 306 has stored thereon logic that, in response to execution by the one or more processors 302, cause the image analysis computing system 202 to provide an image collection engine 310, an OAC engine 312, a segmentation engine 314, a measurement engine 316, a training engine 318, and a prediction engine 322.
As used herein, “computer-readable medium” refers to a removable or nonremovable device that implements any technology capable of storing information in a volatile or non-volatile manner to be read by a processor of a computing device, including but not limited to: a hard drive; a flash memory; a solid state drive; random-access memory (RAM); read-only memory (ROM); a CD-ROM, a DVD, or other disk storage; a magnetic cassette; a magnetic tape; and a magnetic disk storage.
In some embodiments, the image collection engine 310 is configured to receive OCT data from the OCT imaging system 204. In some embodiments, the image collection engine 310 may also be configured to collect training images from one or more storage locations, and to store the training images in the image data store 308. In some embodiments, the OAC engine 312 is configured to calculate OAC data based on OCT data received from the OCT imaging system 204.
In some embodiments, the training engine 318 is configured to train one or more machine learning models to label areas of geographic atrophy depicted in at least one of OAC data and OCT data. In some embodiments, the segmentation engine 314 is configured to use suitable techniques to label areas of geographic atrophy in images collected by the image collection engine 310 and/or OAC data generated by the OAC engine 312. In some embodiments, the techniques may include using machine learning models from the model data store 320 to automatically label images. In some embodiments, the techniques may include receiving labels manually entered by expert reviewers.
In some embodiments, the measurement engine 316 is configured to measure one or more attributes of an eye depicted in OAC data. In some embodiments, the prediction engine 322 is configured to predict an enlargement rate of geographic atrophy for an eye based on the attributes measured by the measurement engine 316.
Further description of the configuration of each of these components is provided below.
As used herein, “engine” refers to logic embodied in hardware or software instructions, which can be written in one or more programming languages, including but not limited to C, C++, C#, COBOL, JAVA™, PHP, Perl, HTML, CSS, Javascript, VBScript, ASPX, Go, and Python. An engine may be compiled into executable programs or written in interpreted programming languages. Software engines may be callable from other engines or from themselves. Generally, the engines described herein refer to logical modules that can be merged with other engines, or can be divided into sub-engines. The engines can be implemented by logic stored in any type of computer-readable medium or computer storage device and be stored on and executed by one or more general purpose computers, thus creating a special purpose computer configured to provide the engine or the functionality thereof. The engines can be implemented by logic programmed into an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or another hardware device.
As used herein, “data store” refers to any suitable device configured to store data for access by a computing device. One example of a data store is a highly reliable, high-speed relational database management system (DBMS) executing on one or more computing devices and accessible over a high-speed network. Another example of a data store is a key-value store. However, any other suitable storage technique and/or device capable of quickly and reliably providing the stored data in response to queries may be used, and the computing device may be accessible locally instead of over a network, or may be provided as a cloud-based service. A data store may also include data stored in an organized manner on a computer-readable storage medium, such as a hard disk drive, a flash memory, RAM, ROM, or any other type of computer-readable storage medium. One of ordinary skill in the art will recognize that separate data stores described herein may be combined into a single data store, and/or a single data store described herein may be separated into multiple data stores, without departing from the scope of the present disclosure.
From a start block, the method 400 advances to block 402, where an image collection engine 310 of an image analysis computing system 202 receives optical coherence tomography data (OCT data) from an OCT imaging system 204. It will be understood that, given the presence of A-lines and B-scans, the OCT data constitutes a volumetric image of the scanned area. The OCT data may be SS-OCT data, SD-OCT data, or any other suitable form of OCT data. In some embodiments, the OCT data includes both A-lines and B-scans. In some embodiments, the OCT data may include 6×6 mm scans with 1536 pixels on each A-line (3 mm), 600 A-lines on each B-scan, and 600 sets of twice repeated B-scans. In some embodiments, scans with a signal strength less than a signal strength threshold (such as 7) or evident motion artifacts may be excluded.
At block 404, an OAC engine 312 of the image analysis computing system 202 calculates an optical attenuation coefficient (OAC) for each pixel of the OCT data to create OAC data corresponding to the OCT data. In some embodiments, the OAC may be calculated for each pixel using a depth-resolved single scattering model. Briefly, if it is assumed that all light is completely attenuated within the imaging range, the backscattered light is a fixed fraction of the attenuated light, and the detected light intensity is uniform over a pixel, then the OAC μ[i] at each pixel i within the volumetric imaging range may be determined by:
At subroutine block 406, a segmentation engine 314 of the image analysis computing system 202 determines an area exhibiting geographic atrophy. Any suitable technique for determining the area exhibiting geographic atrophy may be used. In some embodiments, an automatic technique for determining the area exhibiting geographic atrophy may be used. One non-limiting example of a technique that uses the OAC data to automatically detect areas exhibiting geographic atrophy using a machine learning model is illustrated in
In some embodiments, a manual technique for determining the area exhibiting geographic atrophy may be used, with the subsequent measurement and prediction steps being performed automatically. If using a manual technique, the segmentation engine 314 may cause images representing the OCT data and/or the OAC data to be presented to a clinician, and the clinician may manually indicate areas exhibiting geographic atrophy via a user interface provided by the segmentation engine 314.
At block 408, the segmentation engine 314 determines an adjacent area that is adjacent to the area exhibiting geographic atrophy. The segmentation engine 314 determines the adjacent area by finding an area that is within a specified area adjacent to the area exhibiting geographic atrophy. Any suitable adjacent area may be used. As one non-limiting example, the adjacent area may be a 1-degree rim region that extends from 0 μm outside of the margin of the area exhibiting geographic atrophy to 300 μm outside of the margin of the area exhibiting geographic atrophy. As another non-limiting example, the adjacent area may be an additional 1-degree rim region that extends from 300 μm outside of the margin of the area exhibiting geographic atrophy to 600 μm outside of the margin of the area exhibiting geographic atrophy. As yet another non-limiting example, the adjacent area may be a 2-degree rim region that extends from 0 μm outside of the margin of the area exhibiting geographic atrophy to 600 μm outside of the margin of the area exhibiting geographic atrophy. As still another non-limiting example, the adjacent area may be an area from 600 μm outside of the margin of the area exhibiting geographic atrophy to the edge of the scan area. As a final non-limiting example, the adjacent area may be an entire area from the margin of the area exhibiting geographic atrophy to the edge of the scan area. In some embodiments, the listed sizes of these areas may be approximate, and may be smaller or larger by 5% (e.g., a region that extends from 0 μm outside of the margin of the area exhibiting geographic atrophy to an amount between 285 μm to 315 μm outside of the margin of the area exhibiting geographic atrophy, etc.).
Returning to
In some embodiments, the one or more attributes may include one or more features based on the measurements, including but not limited to one or more of a mean of the measurements within the adjacent area and a standard deviation of the measurements within the adjacent area. In some embodiments, different attributes may be measured within different adjacent areas (for example, a first attribute may be measured in a first adjacent area, while a second attribute is measured in a second adjacent area).
At block 412, a prediction engine 322 of the image analysis computing system 202 generates a predicted enlargement rate based on the one or more attributes within the adjacent area. In some embodiments, the prediction engine 322 retrieves a prediction model from the model data store 320 that corresponds to the adjacent area and the one or more measured attributes, and uses the prediction model to generate the predicted enlargement rate. In some embodiments, the prediction model may be a multiple linear regression model that uses one or more attributes measured in one or more adjacent areas as input, and that outputs a predicted enlargement rate. Two non-limiting examples of prediction models are described below in Example One and Example Two.
At block 414, the image analysis computing system 202 provides the predicted enlargement rate for use in at least one of diagnosis, determining an appropriate treatment, and evaluating an applied treatment. By being able to automatically predict an enlargement rate using the prediction model, a subject can be advised about the severity of their AMD and the urgency of treatment without needing to wait to observe the actual progression of the condition. Further, the efficacy of applied treatments can be evaluated without having to wait to observe the effects over long periods of time, and can instead be evaluated during or shortly after the course of treatment, thus improving the efficacy of the treatment.
The method 400 then proceeds to an end block and terminates.
From a start block, the procedure 600 advances to block 602, where the segmentation engine 314 identifies a location of a Bruch's membrane 106 based on the OCT data. In some embodiments, a manufacturer of the OCT imaging system 204 may provide an engine for identifying the location of the Bruch's membrane 106, and the engine may be executed by the OCT imaging system 204 or the segmentation engine 314. In some embodiments, the manufacturer of the OCT imaging system 204 may provide logic for identifying the location of the Bruch's membrane 106, and the logic may be incorporated into the segmentation engine 314. One non-limiting example of such an engine is provided by Carl Zeiss Meditec, of Dublin, CA. In some embodiments, similar techniques may be used to identify the locations of other structures within the OCT data, including but not limited to a lower boundary of a retinal nerve fiber layer (RNFL).
At block 604, the segmentation engine 314 uses the location of the Bruch's membrane 106 indicated by the OCT data to determine the location of the Bruch's membrane 106 in the OAC data. Since the OAC data is derived from the OCT data as described at block 404, the location of each volumetric pixel in the OAC data corresponds to a location of a volumetric pixel in the OCT data. Accordingly, the determined location of the Bruch's membrane 106 (and/or other detected structures) from the OCT data may be transferred to the corresponding locations in the OAC data.
At block 606, the segmentation engine 314 extracts a slab of the OAC data located above the Bruch's membrane 106. In some embodiments, the extracted slab of the OAC data may extend from the Bruch's membrane 106 to the RNFL. In some embodiments, the extracted slab of the OAC data may be a predetermined thickness, such as extending from the Bruch's membrane 106 to a predetermined distance above the Bruch's membrane 106. In one non-limiting example embodiment, the predetermined distance may be a value within a range of 540 μm to 660 μm, such as 600 μm.
At block 608, the segmentation engine 314 generates an en face OAC maximum projection image, an en face OAC sum projection image, and an en face RPE to BM distance map for the slab. The en face OAC maximum projection image represents maximum OAC values through the depth of the slab for each given pixel. The en face OAC sum projection image represents a sum of the OAC values through the depth of the slab for each given pixel. The en face RPE-BM distance map represents a measured distance between the retinal pigment epithelium 104 and the Bruch's membrane 106 at each pixel. In some embodiments, the location of the retinal pigment epithelium 104 may be determined by the pixel with the maximum OAC value above the Bruch's membrane 106 along each A-line.
At block 610, the segmentation engine 314 creates a false color image by combining the en face OAC maximum projection image, the en face OAC sum projection image, and the en face RPE to BM distance map. Each image may be assigned to a color channel for the false color image in order to combine the separate images. For example, the value for a pixel for the en face OAC maximum projection image may be assigned to the red channel, the value for a corresponding pixel from the en face OAC sum projection image may be assigned to the green channel, and the value for a corresponding pixel from the en face RPE to BM distance map may be assigned to the blue channel.
In some embodiments, the values of the separate images may be assigned to specific dynamic ranges in order to normalize the values for the false color image. As one non-limiting example, the values in the en face OAC maximum projection image may be assigned to a dynamic range of 0 to 60 mm−1, the values in the en face OAC sum projection image may be assigned to a dynamic range of 0 to 600 (unitless), and the values in the en face RPE to BM distance map may be assigned to a dynamic range of 0 to 100 μm. In some embodiments, different dynamic ranges may be used, including dynamic ranges of other units and dynamic ranges with upper bounds that vary by up to 10% from the listed values above. In some embodiments, a smoothing filter may be applied to the false color image to reduce noise. One example of a suitable smoothing filter to be used is a 5×5 pixel median filter, though in other embodiments, other smoothing filters may be used.
At block 612, the segmentation engine 314 provides the false color image as input to a machine learning model trained to determine the area exhibiting geographic atrophy. In some embodiments, the false color image may be resized to match a dimension of an input layer of the machine learning model. Any suitable machine learning model may be used that accomplishes the segmentation task of the false color image (that is, that provides an identification of whether or not each pixel represents a location of geographic atrophy), including but not limited to an artificial neural network. One non-limiting example of a suitable machine learning model is a U-net, and a non-limiting example of an architecture of a suitable U-net and techniques for training are illustrated in
The procedure 600 then proceeds to an end block, where the segmentation that constitutes an indication of areas that exhibit geographic atrophy in the OAC data is returned to the procedure's caller, and the procedure 600 terminates.
From a start block, the procedure 700a advances to block 702, where the measurement engine 316 identifies a Bruch's membrane 106 location in the OAC data. In some embodiments, techniques similar to those described in block 602 may be used to identify the Bruch's membrane 106 location. In some embodiments, the Bruch's membrane 106 location previously determined at block 602 may be reused by block 702.
At block 704, the measurement engine 316 identifies a retinal pigment epithelium 104 location in the OAC data. In some embodiments, the retinal pigment epithelium 104 location may be identified by the pixel with the maximum OAC value above the Bruch's membrane 106 location along each A-line.
At block 706, the measurement engine 316 applies a smoothing filter to the Bruch's membrane 106 location and the retinal pigment epithelium 104 location. In some embodiments, a 5×5 pixel median filter may be used for the smoothing.
At block 708, the measurement engine 316 determines one or more characteristics of a distance between the smoothed Bruch's membrane 106 location and the smoothed retinal pigment epithelium 104 location within the adjacent area. Any suitable characteristics of the distance may be used. In some embodiments, a mean of the distance within the adjacent area may be used. In some embodiments, a standard deviation of the distance within the adjacent area may be used. In some embodiments, other statistical characteristics of the distance within the adjacent area may be used as attributes.
At block 710, the measurement engine 316 provides the one or more characteristics as the measured RPE-BM distance attribute for the adjacent area. The procedure 700a then advances to an end block and terminates.
From a start block, the procedure 700b advances to block 712, where the measurement engine 316 identifies an outer plexiform layer 120 location in the OAC data. In some embodiments, the upper boundary of the outer plexiform layer 120 may be detected using a known semi-automated segmentation technique, such as the technique described in Yin X, Chao J R, Wang R K; User-guided segmentation for volumetric retinal optical coherence tomography images; J Biomed Opt. 2014; 19(8):086020; doi: 10.1117/1.JBO.19.8.086020, the entire disclosure of which is hereby incorporated by reference herein for all purposes.
At block 714, the measurement engine 316 identifies a retinal pigment epithelium 104 location in the OAC data. As discussed above with respect to block 704, the retinal pigment epithelium 104 location may be identified by the pixel with the maximum OAC value above the Bruch's membrane 106 location along each A-line.
At block 716, the measurement engine 316 applies a smoothing filter to the outer plexiform layer 120 location and the retinal pigment epithelium 104 location. As discussed above, the smoothing filter may be a 5×5 pixel median filter, which may be applied to the B-scan of the OAC data.
At block 718, the measurement engine 316 determines one or more characteristics of a distance between the smoothed outer plexiform layer 120 location and the smoothed retinal pigment epithelium 104 location in the adjacent area. As with the characteristics of the RPE-BM distance, any suitable characteristics of the distance between the smoothed outer plexiform layer 120 location and the smoothed retinal pigment epithelium 104 location may be used as the characteristics, including but not limited to a mean, a standard deviation, or combinations thereof.
At block 720, the measurement engine 316 provides the one or more characteristics as the measured outer retinal thickness attribute for the adjacent area. The procedure 700b then advances to an end block and terminates.
Another non-limiting example of an attribute that may be useful in generating predicted enlargement rates is a choriocapillaris flow deficit. One of ordinary skill in the art will recognize that techniques are available for measuring choriocapillaris flow deficits from swept-source OCT angiography (SS-OCTA) images, such as those described in Thulliez, M., Zhang, Q., Shi, Y., Zhou, H., Chu, Z., de Sisternes, L., Durbin, M. K., Feuer, W., Gregori, G., Wang, R. K., & Rosenfeld, P. J. (2019); Correlations between Choriocapillaris Flow Deficits around Geographic Atrophy and Enlargement Rates Based on Swept-Source OCT Imaging; Ophthalmology. Retina, 3(6), 478-488; https://doi.org/10.1016/j.oret.2019.01.024, the entire disclosure of which is hereby incorporated by reference herein for all purposes.
Briefly, detection of angiographic flow information may be achieved using the complex optical microangiography (OMAGC) technique. Choriocapillaris (CC) en face flow images may be generated by applying a 15 μm thick slab with the inner boundary located 4 μm under the Bruch's membrane 106. Retinal projection artifacts may be removed prior to compensating the CC en face flow images for signal attenuation caused by overlying structures such as RPE abnormalities including drusen, hyperreflective foci, and/or RPE migration. Compensation may be achieved by using the inverted images that corresponded to the CC en face structural images. The CC images may then undergo thresholding to generate CC flow deficit (FD) binary maps. Small areas of CC FD (e.g., CC FDs with a diameter smaller than 24 μm) may be removed as representing physiological FDs and speckle noise before final CC FD calculations.
Once CC FD areas have been labeled, various characteristics of the CC FD may be measured as attributes for an adjacent area. For example, a percentage of FDs (CC FD %) may be used, which is a ratio of the number of all pixels representing FDs divided by all of the pixels within the adjacent area. As another example, a mean or averaged FD size (MFDS) may be used, which is an average area of all isolated regions representing CC FDs within the adjacent area.
In the machine learning model 802, a 512×512 input layer accepts the three-channel false color image as input. As illustrated, the machine learning model 802 is shown as accepting either a one-channel image or a three-channel false color image as input. In some embodiments, the machine learning model 802 may be trained to accept a single-channel en face image for a slab extracted from the OCT data. For example, a subRPE slab extending from 64 μm below the Bruch's membrane 106 to 400 μm below the Bruch's membrane 106 may be extracted from the OCT data, and an en face image may be created using the sum projection for providing to a one-channel input layer of the machine learning model 802. Separate machine learning models 802 may be trained for the three-channel input layer and the one-channel input layer, and their performance may be compared.
In the contracting path (the left side of the machine learning model 802), the input layer is followed by two 3×3 convolutional layers with batch normalization and ReLU, a 2×2 MaxPool, two 3×3 convolutional layers with batch normalization and ReLU, another 2×2 MaxPool, two more 3×3 convolutional layers with batch normalization and ReLU, and a final 2×2 MaxPool. The bottom layer of the U-net includes two 3×3 convolutional layers, with batch normalization and ReLU, followed by a 2×2 up-convolution with ReLU. The results of the contracting path are copied and concatenated to the expansive path (the right side of the machine learning model 802).
In the expansive path of the machine learning model 802, a 3×3 convolutional layer with dropout, batch normalization, and ReLU is followed by a 3×3 convolution layer with batch normalization and ReLU and then a 2×2 up-convolution with ReLU. Next, a 3×3 convolution layer with dropout, batch normalization and ReLU is followed by another 3×3 convolution layer with batch normalization and ReLU and a 2×2 up-convolution with ReLU. Another 3×3 convolution layer with dropout, batch normalization and ReLU is executed, followed by another 3×3 convolution layer with batch normalization and ReLU and a 2×2 up-convolution with ReLU. Finally, a 3×3 convolution layer with dropout, batch normalization, and ReLU is followed by a 3×3 convolution layer with batch normalization and ReLU, and then a 1×1 convolution layer with a sigmoid activation function produces the segmented output.
The following description describes a non-limiting example of a process of training a machine learning model 802 that was used to study the performance of the machine learning model 802. One of ordinary skill in the art will recognize that the example training steps described below should not be seen as limiting, and that in some embodiments, other steps (including but not limited to training data generated, selected, and organized using other techniques; different initializers, optimizers, evaluation metrics, and/or loss functions; and different settings for various constants and numbers of epochs) may be used.
Two machine learning models 802 were trained using the illustrated architecture but different input layers: one with a three-channel input layer to accept the false color images based on the OAC data as described above, and another with a one-channel input layer to accept the en face images of the subRPE slab from the OCT data. The en face images of the subRPE slab from the OCT data have been used in previous studies, and are being used with the novel machine learning model 802 in the present study to both show the superiority of the machine learning model 802 independent of the images used, and also to provide an apples-to-apples comparison to illustrate the superiority of the use of the described false color images based on OAC data compared to the previously used en face subRPE slab images generated from OCT data.
Training data was created and stored in the image data store 308 by manually annotating areas of geographic atrophy in the en face images of the subRPE slab from the OCT data, referencing B-scans, and was retrieved from the image data store 308 by the training engine 318 to conduct the training process.
Training used 80% of all eyes, and testing used 20% of the eyes. Within the training cases, an 80:20 split between training and validation was applied, partitioned at the eye level. Cases were shuffled and the set division was random. The learning rate, dropout, and batch normalization hyperparameters for the training process were tuned on the validation set using grid search. Data augmentation with zoom, shear, and rotation was used, and a batch size of 8 was used. For each 3×3 convolution layer, the He normal initializer was used for kernel initialization. The Adam optimizer was used and the model evaluation metric was defined as the soft DSC (sDSC). The loss function was the sDSC loss:
To evaluate the performance of the trained models, DSC, area square-root difference (ASRD), subject-wise sensitivity, and specificity were calculated on the testing set:
To further compare the identified GA regions, total area and square-root area measurements of GA were calculated for both ground truth and model outputs. A square-root transformation was applied to calculate the size and growth of geographic atrophy since this strategy decreases the influence of baseline lesion size on the test-retest variability and on the growth of geographic atrophy. The paired t-test was used to compare model outputs using the false color images based on the OAC data and the subRPE images based on the OCT data. Pearson's linear correlation was used to compare the square-root area of the manual and automatic segmentations, and Bland-Altman plots were used to analyze the agreement between the square-root area of the manual and automatic segmentations. P values of <0.05 were considered to be statistically significant.
In total, 80 eyes diagnosed with geographic atrophy secondary to nonexudative AMD and 60 normal eyes with no history of ocular disease, normal vision, and no identified optic disc, retinal, or choroidal pathologies on examination were included in the study. All cases were randomly shuffled such that 51 geographic atrophy eyes and 38 normal eyes were used for training, 13 geographic atrophy eyes and 10 normal eyes were used for validation, and 16 geographic atrophy eyes and 12 normal eyes were used for testing. In the training dataset, 22 out of these 51 eyes had three scans from three visits and these scans were added into the training set for data augmentation. Eyes in the validation and testing set only had one scan.
Both models were trained using the same learning rate of 0.0003 and the same batch normalization momentum of 0.1 with the scale set as false. A dropout of 0.3 was used for the machine learning model 802 trained to process the false color images and a dropout of 0.5 was used for the machine learning model 802 trained to process the single-channel images based on the OCT data. All hyperparameters were tuned on the validation set. Each model was trained with 200 epochs and their specific sDSC for training, validation, and testing are given in the following table:
A series of evaluation metrics were quantified on the testing cases for each trained model, and their specific values are tabulated in the following table:
For testing, the model outputs, geographic atrophy probability maps (0-1), were binarized with a threshold of 0.5. DSC was calculated for each individual image and the mean and standard deviation (SD) were reported in the table above for each model. In the 16 geographic atrophy eyes in the testing set, the machine learning model 802 operating on the false color images from the OAC data significantly outperformed the machine learning model 802 operating on the subRPE slab from the OCT data (p=0.03, paired t-test). Both models achieve 100% sensitivity and 100% specificity in identifying geographic atrophy subjects from normal subjects.
To further compare the quantification of segmentation generated by our models with the ground truth, the geographic atrophy square-root area was calculated for all geographic atrophy cases in the test set.
Geographic atrophy square-root area segmented by both models showed significant correlation with ground truth (R2=0.99 for OAC data model and R2=0.92 for OCT data model, both p<0.0001). Both model outputs also showed satisfactory agreement with the ground truth. The machine learning model 802 operating on the false color images from the OAC data resulted in a smaller bias of 11 μm while the machine learning model 802 operating on the subRPE slab from the OCT data resulted in a larger bias of 117 μm, compared with the ground truth.
Using the same model architecture, the same hyper-parameter tuning process, and the same patients' OCT scans, the above demonstrates a significantly higher agreement with the ground truth by using the machine learning model 802 trained to use the false color images generated from OAC data than by using subRPE images generated from OCT data. For all 28 eyes in the testing sets, both models successfully identified eyes with geographic atrophy from normal eyes. For the 16 eyes with geographic atrophy in the testing sets, the machine learning model 802 trained to process false color images generated from OAC data achieved a mean DSC of 0.940 and a SD of 0.032, significantly higher than the other model with a mean DSC of 0.889 and a SD of 0.056 (p=0.03, paired t-test), respectively. For geographic atrophy square-root area measurements, the machine learning model 802 trained to process false color images generated from OAC data achieved a stronger correlation with the ground truth than the other model (r=0.995 vs r=0.959, r2=0.99 vs r2=0.92), as well as a smaller mean bias (11 μm vs 117 μm).
That said, using the machine learning model 802 with the subRPE images generated from SS-OCT data, a DSC of 0.889±0.056 was obtained, similar to what were used in previous SD-OCT studies. Though different datasets were used in different studies and direct comparisons of testing DSC values are somewhat unfair, the machine learning model 802 trained on the OCT data achieved a segmentation accuracy that was similar to these previous studies. That said, the machine learning model 802 trained to process false color images generated from OAC data achieved a significantly higher segmentation accuracy (0.940±0.032) compared with the similar machine learning model 802 using OCT subRPE images. This is a fair comparison since the same volumetric OCT data was used to generate the en face images for input in the models, though the OAC data undergoes further preprocessing. It should also be noted that though the structure of the machine learning model 802 is simpler compared to previously published studies, the segmentation accuracy provided by the machine learning model 802 in terms of DSC is similar to or superior to what were reported in previous studies, possibly due to the use of the enhanced contrast of geographic atrophy produced by using the OAC.
In some embodiments, the distance between the retinal pigment epithelium 104 and the Bruch's membrane 106 (RPE-BM distance) may be one of the attributes measured within the adjacent area, and may be used for prediction of progression of geographic atrophy. In this example, a multiple linear regression model that accepts RPE-BM distance as well as choriocapillaris flow deficit percentage (CC FD %) serves as the prediction model for generating the predicted enlargement rate.
In a study, Pearson correlation was used to evaluate the relationships between the OAC-measured RPE-BM distances and the normalized annual square root enlargement rates of geographic atrophy, as well as the relationship between previously determined choriocapillaris flow deficit percentages (CC FD %) and the RPE-BM distance of the same eyes. To assess the combined effects of RPE-BM distance and CC FD % on predicting geographic atrophy growth, a multiple linear regression model was calculated using RPE-BM distance and CC FD % as variables and the normalized annual square root enlargement rate of geographic atrophy as the outcome. A P value of <05 was considered to be statistically significant.
A total of 38 eyes from 27 subjects diagnosed with geographic atrophy secondary to nonexudative AMD were included in the study. The relationship between the enlargement rate of geographic atrophy in these eyes and the surrounding CC FD % s and underlying choroidal parameters were previously determined in these eyes. The techniques illustrated in
For the 38 eyes, the annual square root enlargement rates ranged from 0.11 mm/y to 0.78 mm/y, with a mean of 0.31 mm/y and a standard deviation of 0.15 mm/y. The RPM-BM distance calculated using the technique illustrated in
A significant correlation between the annual square root enlargement rates of geographic atrophy and CC FD % in these same eyes had previously been determined. To further understand the relationships between CC FD % and RPE-BM distances, Pearson's correlation was performed between these two metrics in each adjacent area, and no significant correlations were found in any adjacent area (all Pearson r<0.083, all P>0.622). Therefore, CC FD % in the total scan area minus GA (strongest correlation for CC FD %) and RPE-BM distance in R1 (strongest correlation for RPE-BM distance) were combined to fit a multiple linear regression model to predict annual square root enlargement rates for geographic atrophy. This prediction model was as follows:
Using these variables, this prediction model resulted in a combined r of 0.75 and r2 of 0.57.
In some embodiments, the outer retinal layer (ORL) thickness may be one of the attributes measured within the adjacent area, and may be used for prediction of progression of geographic atrophy. In this example, a multiple linear regression model that accepts ORL thickness, as well as the RPE-BM distance and choriocapillaris flow deficit percentage (CC FD %) discussed in Example One, serves as the prediction model for generating the predicted enlargement rate.
In a study of the same eyes as Example One, Pearson's correlation was used to evaluate the relationships between the ORL thickness (measured using the procedure 700b illustrated in
A P value of <0.05 was considered to be statistically significant. The below table shows the detailed correlations (r) and significance values (P) for each adjacent area and the averaged ORL thickness in each sub-region. The ORL thickness measurements in all adjacent areas except for R3 were shown to have significant negative correlations with the annual square root enlargement rate of geographic atrophy. The R1 region had the strongest negative correlation (r=−0.457, P=0.004) among all of the adjacent areas. The correlations in all adjacent areas are shown as scatter plots in
Adding the ORL thickness measurement in R1 (the strongest correlation with annual square root enlargement rate of geographic atrophy) to the prediction model that already considered CC FD % and RPE-BM thickness provided an improvement in r to 0.79 (r2=0.62). The predicted enlargement rates calculated by this prediction model, with a mean±SD of 0.32 mm/year±0.12 mm/year and 95% confidence intervals ranging from 0.277 mm/year to 0.357 mm/year, significantly correlated with (P=0.028) the measured annual square root enlargement rates of geographic atrophy (mean±SD of 0.31 mm/year±0.15 mm/year, with 95% confidence intervals ranging from 0.267 mm/year to 0.368 mm/year).
A Pearson's correlation was further performed between CC FD % and ORL thickness and between RPE-BM distance and ORL thickness in each adjacent area. No significant correlations were found in any adjacent areas between CC FD % and ORL thickness, but a significant correlation was found in the R1 region between RPE-BM distance and ORL thickness (Pearson's r=−0.398, P=0.013).
While illustrative embodiments have been illustrated and described, it will be appreciated that various changes can be made therein without departing from the spirit and scope of the invention.
This application claims the benefit of Provisional Application No. 63/182,328, filed Apr. 30, 2021, the entire disclosure of which is hereby incorporated by reference herein for all purposes.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/US2022/027002 | 4/29/2022 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 63182328 | Apr 2021 | US |
