Patent Grant
10769789
Field of the Disclosure
The present disclosure relates to an image processing apparatus and image processing method for processing an image of a subject.
Description of the Related Art
Optical coherence tomography (hereinafter referred to OCT) has been used as a non-destructive and non-invasive method for obtaining a tomographic image of a measurement target, such as a living body. OCT is widely used especially in the field of ophthalmology to obtain tomographic images of a retina in a fundus of an eye to be examined for ophthalmic examination of the retina and the like.
In OCT, light reflected from a measurement target interferes with light reflected from a reference mirror, and the time- or wavenumber-dependence of the intensity of the interfering light is analyzed to obtain tomographic images. Examples of a known apparatus for obtaining such an OCT image include a time domain OCT, a spectral domain OCT (SD-OCT), and a swept source OCT (SS-OCT). The time domain OCT obtains depth information about a measurement target by moving a reference mirror to different positions. The SD-OCT uses a broad-bandwidth light source. The SS-OCT uses as a light source a wavelength-tunable light source in which an oscillation wavelength is tunable. The SD-OCT and the SS-OCT are collectively referred to as Fourier domain OCT (FD-OCT).
In recent years, a virtual angiography using FD-OCT has been discussed, and such virtual angiography is called OCT angiography (OCTA). A fluorescein angiography, which is a commonly-used angiography in modern clinical practice, requires injection of a fluorescent dye (e.g., fluorescein or indocyanine green) into a body and produces a two-dimensional representation of blood vessels through which the fluorescent dye passes. On the other hand, the OCTA enables a non-invasive virtual angiography and can provide a three-dimensional representation of a network of blood flow. Furthermore, the OCTA has a higher resolution than the fluorescein angiography and can produce images of fine blood vessels or blood flow in the eye fundus. For this reason, the OCTA has been attracting rising attention.
A method of detecting blood flow with the OCTA is discussed in Fingler et al. “Mobility and transverse flow visualization using phase variance contrast with spectral domain optical coherence tomography”, Optics Express, Vol. 15, No. 20, pp 12637-12653 (2007). In the method discussed therein, only time-modulated signals are extracted from OCT signals to separate the OCT signals that are from blood flow. Other methods for detecting blood flow are also discussed including a method which uses intensity fluctuations due to blood flow (United States Patent Application Publication No. 2014/221827) and a method which uses phase fluctuations due to blood flow. As used herein, an image showing time-modulated signals which are signals among OCT signals is sometimes referred to as a “motion contrast image”, a pixel value of the motion contrast image as “motion contrast”, and a data set of the motion contrast as “motion contrast data”.
Meanwhile, a polarization OCT developed as a functional OCT is capable of visualizing structural information, such as a nerve fiber layer and a retinal layer. Investigative Ophthalmology & Visual Science, Jan. 7, 2013, Zotter S et al., “Measuring retinal nerve fiber layer birefringence, retardation, and thickness using wide-field, high-speed polarization sensitive spectral domain OCT” discusses a technique for obtaining en face maps using the polarization OCT by integrating, along a thickness direction, three-dimensional data per unit thickness on a retinal nerve fiber layer (RNFL) deflection parameter called retardation.
According to an aspect of the present invention, an image processing apparatus includes an information obtaining unit configured to obtain three-dimensional polarization sensitive tomographic information and three-dimensional motion contrast information about a subject based on tomographic signals of lights having different polarizations, the lights obtained by splitting a combined light obtained by combining a returned light from the subject illuminated with a measurement light with a reference light corresponding to the measurement light, an obtaining unit configured to obtain a lesion region of the subject using the three-dimensional polarization sensitive tomographic information, and an image generation unit configured to generate an image in which the lesion region is superimposed on a motion contrast image generated using the three-dimensional motion contrast information.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
In age-related macular degeneration (AMD) involving development of new blood vessels, the new blood vessels which are characteristic of AMD grow rapidly after developing from the choroid, breaking through the Bruch's membrane, and forming underneath or above the retinal pigment epithelium (RPE). A leakage of blood from the new blood vessels causes distortion of vision and formation of fibrotic tissues around the leakage point. In order to provide appropriate treatment to a patient with AMD involving development of new blood vessels, the new blood vessels and fibrotic tissues (fibrosis) need to be discriminated from each other and identified. Because, for example, to the blood leakage from the new blood vessels, administration of a drug which inhibits a vascular endothelial growth factor (VEGF) can inhibit the growth of the new blood vessels but is not likely to have an effect on tissues that are already fibrotic. Further, in order to determine the effect of the treatment, it is necessary to identify changes in the respective tissues over time.
However, in conventional OCT luminance images, it is difficult to discriminate between RPE, fibrotic tissues, and new blood vessels because all of them appear as similar high luminance reflection regions. Further, with OCTA or polarization OCT alone, it is difficult to comprehensively identify tissues associated with the development of AMD. Further, it is also difficult to identify temporal or spatial changes in the tissues. Thus, it has been difficult to find out the development of the disease.
The present invention is directed to a technique for generating an image in which tissues that are difficult to identify, such as new blood vessels and fibrotic tissues, are identifiable.
An imaging apparatus according to an exemplary embodiment of the present invention will be described below with reference to the drawings. Configurations and processes described in the below-described exemplary embodiment are mere examples and are not intended to limit the scope of the invention. Further, while a subject is a human eye (fundus) in the present exemplary embodiment, the subject is not limited to the human eye (fundus) and can be, for example, skin. Further, while an imaging target is the fundus of an eye in the present exemplary embodiment, the imaging target can be an anterior eye.
<Configuration of Polarization OCT Apparatus>
First, a configuration of the OCT image obtaining unit 800 will be described. A light source 801 is a swept source (SS) light source which emits light while sweeping at, for example, a sweep central wavelength of 1050 nm and a sweep width of 100 nm. The light emitted from the light source 801 is guided to a beam splitter 810 via a single mode fiber (hereinafter, “SM fiber”) 802, a polarization control device 803, a connector 804, a SM fiber 805, a polarizer 806, a polarization maintaining (PM) fiber (hereinafter, “PM fiber”) 807, a connector 808, and a PM fiber 809, and split into measurement light (also referred to as “OCT measurement light”) and reference light (also referred to as “reference light corresponding to OCT measurement light”). The splitting ratio of the beam splitter 810 is 90 (reference light): 10 (measurement light). The polarization control device 803 can change polarization of the light emitted from the light source 801 into a desired polarization state. The polarizer 806 is an optical element having the characteristic that it transmits only specific linearly-polarized components. In general, the light emitted from the light source 801 has a high degree of polarization, and a majority of the light has a specific polarization direction, but the light also includes light having no specific polarization direction, which is called randomly-polarized components. The randomly-polarized components are known to impair the image quality of polarization OCT images, so the polarizer 806 cuts the randomly-polarized components. Only light that is in a specific linearly-polarized state can pass through the polarizer 806, so the polarization control device 803 adjusts the polarization state such that a desired amount of light enters an eye to be examined 118.
The split measurement light is output via a PM fiber 811 and then changed into parallel light by a collimator 812. The measurement light changed to the parallel light passes through a quarter-wave plate 813 and then enters the eye to be examined 118 via a galvano scanner 814, a scan lens 815, and a focusing lens 816. The galvano scanner 814 is a scanner which scans a fundus Er of the eye to be examined 118 with the measurement light. While the galvano scanner 814 is described above as a single mirror, the actual galvano scanner 814 includes two galvano scanners perform raster scanning on the fundus Er of the eye to be examined 118. Further, the focusing lens 816 is fixed onto a stage 817 and is moved along an optical axis direction to adjust a focus. The galvano scanner 814 and the stage 817 are controlled by the signal obtaining control unit 145, and a desired range (also referred to as “tomographic image obtaining range”, “tomographic image obtaining position”, “measurement light illumination position”) of the fundus Er of the eye to be examined 118 can be scanned with the measurement light. Further, the quarter-wave plate 813 is an optical element having the characteristic that it delays by a quarter wavelength a phase between an optical axis of the quarter-wave plate 813 and an axis which is orthogonal to the optical axis. In the present exemplary embodiment, the optical axis of the quarter-wave plate 813 is rotated about the optical axis by degrees with respect to the direction of the linear polarization of the measurement light emitted from the PM fiber 811 so that circularly polarized light enters the eye to be examined 118. While details are not described in the present exemplary embodiment, a tracking function is desirably provided. The tracking function is a function with which the movement of the fundus Er is detected and the mirror of the galvano scanner 814 follows the movement of the fundus Er to scan the fundus Er. The tracking can be performed using a commonly-used technique, either in real time or post-processing. Examples include a method using a scanning laser ophthalmoscope (SLO). In this method, two-dimensional images of a plane of the fundus Er which is vertical to the optical axis are obtained over time with the SLO, and a feature portion such as a vascular bifurcation in the images is obtained. Then, how the feature point in the obtained two-dimensional images is moved is calculated as an amount of movement of the fundus Er, and the calculated amount of movement is fed back to the galvano scanner 814 to perform real-time tracking.
The measurement light is caused to enter the eye to be examined 118 and is focused onto the fundus Er by the focusing lens 816 placed on the stage 817. The measurement light illuminating the fundus Er is reflected and scattered by each retinal layer and then returns through the above-described optical path to the beam splitter 810. The returned light of the measurement light having entered the beam splitter 810 passes through a PM fiber 826 and enters a beam splitter 828.
On the other hand, the reference light split at the beam splitter 810 is output via a PM fiber 819 and then changed into parallel light by a collimator 820. The reference light enters a PM fiber 827 via a half-wave plate 821, a dispersion compensation glass 822, a ND filter 823, and a collimator 824. The collimator 824 and one end of the PM fiber 827 are fixed onto a coherence gate stage 825 and are controlled by the signal obtaining control unit 145 to drive along the optical axis direction correspondingly to a difference in axial length of the eye of an examinee, etc. The half-wave plate 821 is an optical element having the characteristic that it delays by a half wavelength a phase between the optical axis of the half-wave plate 821 and an axis which is orthogonal to the optical axis. In the present exemplary embodiment, an adjustment is made such that the linear polarization of the reference light output from the PM fiber 819 is in a polarization state in which a longer axis is inclined by 45 degrees at the PM fiber 827. While the optical length of the reference light is changed in the present exemplary embodiment, any method to change the difference in optical length between the optical paths of the measurement light and the reference light can be employed.
The reference light having passed through the PM fiber 827 enters the beam splitter 828. The beam splitter 828 combines the returned light of the reference light with the reference light to obtain combined light (interfering light) and then splits the combined light into two. The split interfering light beams have opposite phases to each other (hereinafter, “positive and negative components”). The positive component of the split interfering light passes through a PM fiber 829, a connector 831, and a PM fiber 833 and then enters a polarization beam splitter 835. On the other hand, the negative component of the interfering light passes through a PM fiber 830, a connector 832, and a PM fiber 834 and then enters a polarization beam splitter 836.
The polarization beam splitters 835 and 836 split the interfering light along two orthogonal polarization axes into two, which are a vertically-polarized component (hereinafter, “V-polarized component”) and horizontally-polarized component (hereinafter, “H-polarized component”). The positive interfering light having entered the polarization beam splitter 835 is split by the polarization beam splitter 835 into two, which are a positive V-polarized component and positive H-polarized component. The split positive V-polarized component passes through a PM fiber 837 and enters a detector 841, and the split positive H-polarized components passes through a PM fiber 838 and enters a detector 842. On the other hand, the negative interfering light having entered the polarization beam splitter 836 is split by the polarization beam splitter 836 into a negative V-polarized component and negative H-polarized component. The negative V-polarized component passes through a PM fiber 139 and then enters the detector 841, and the negative H-polarized component passes through a PM fiber 840 and then enters the detector 842.
Each of the detectors 841 and 842 is a difference detection device. When two interference signals having phases inverted by 180 degrees are input, the detectors 841 and 842 remove direct current components and output only interference components.
The V-polarized components of the interference signals detected by the detector 841 and the H-polarized components of the interference signals detected by the detector 842 are respectively output as electric signals corresponding to light intensity and input to the signal processing unit 144, which is an example of a tomographic image generation unit.
<Scan Pattern>
As used herein, “A-scan” refers to obtaining of information about a tomographic image of the eye to be examined 118 along a depth direction. Further, “B-scan” refers to a scan for obtaining of information about a tomographic image of the eye to be examined 118 along a direction that is orthogonal to the scanning direction of the A-scan, i.e., a scan for obtaining a two-dimensional image. Further, “C-scan” refers to a scan in a direction that is orthogonal to the tomographic image obtained by the B-scan. Specifically, in two-dimensional raster scanning within a fundus plane during obtaining of a three-dimensional tomographic image, the B-scan refers to a high-speed scanning direction, and the C-scan refers to a low-speed scanning direction in which the B-scans are arranged in the orthogonal direction. By performing the A- and B-scans, a two-dimensional tomographic image is obtained. By performing the A-, B-, and C-scans, a three-dimensional tomographic image is obtained. The B- and C-scans are performed by the galvano scanner 114.
An X-axis scanner 114a (not illustrated) and a Y-axis scanner 114b (not illustrated) respectively include deflection mirrors located such that rotation axes of the deflection mirrors are orthogonal to each other. The X-axis scanner 114a performs scans in an X-axis direction, and the Y-axis scanner 114b performs scans in a Y-axis direction. Each of the X-axis and Y-axis directions is vertical to an eye axis direction of the eyeball, and the X-axis and Y-axis directions are vertical to each other. Further, the direction of a line scan, such as the B- or C-scan, does not have to match the X-axis or Y-axis direction. Thus, the direction of the line scan, such as the B- or C-scan, can be determined as appropriate to a two- or three-dimensional tomographic image to be captured.
Next, an example of a scan pattern according to the present exemplary embodiment will be described below with reference to
In
Further, in
On the other hand, setting the x- and y-pitches larger than ½ of the beam spot diameter on the fundus decrease the definition, but an image of a wider range can be obtained with a small data amount. The x- and y-pitches can be changed freely according to a clinical need.
<Configuration of Control Unit 143>
The following describes a configuration and function of the control unit 143 for controlling the entire apparatus. The control unit 143 includes the signal processing unit 144, the signal obtaining control unit 145, the display unit 146, and the display control unit 149. Further, the signal processing unit 144 includes the image generation unit 147 and the map generation unit 148. The signal processing unit 144 is an example of an information obtaining unit which obtains three-dimensional polarization sensitive tomographic information and three-dimensional motion contrast information on a subject. Further, the image generation unit 147 has a function of generating a luminance image and a motion contrast image from electric signals (interference signals) transmitted from the detectors 841 and 842, and the map generation unit 148 has a function of generating layer information (segmentation of retina) from the luminance image.
Further, the image generation unit 147 has a function of generating a polarization property image from electric signals transmitted to the signal processing unit 144, and the map generation unit 148 has a function of generating a fibrotic tissue image.
The signal obtaining control unit 145 controls the respective components as described above. The signal processing unit 144 generates an image based on the signals output from the detectors 841 and 842, analyzes the generated image, and generates analysis result visualization information.
The image and the analysis result information generated by the signal processing unit 144 are transmitted to the display control unit 149, and the display control unit 149 displays the image and the analysis result on a display screen of the display unit 146. The display unit 146 is, for example, a liquid crystal display. The image data generated by the signal processing unit 144 can be transmitted to the display control unit 149 and then transmitted via wires or wirelessly to the display unit 146. Further, while the display unit 146 and the like are included in the control unit 143 in the present exemplary embodiment, the present exemplary embodiment is not limited to this configuration, and the display unit 146 and the like can be provided separately from the control unit 143 and can be, for example, a tablet which is an example of an apparatus movable by an examiner. In this case, the display unit 146 desirably includes a touch panel function to enable operations on the touch panel for moving the display position of an image, enlarging or reducing the image, changing the displayed image, etc.
<Signal Processing>
The following describes main signal processing performed by the control unit 143, with reference to
[Generation of Tomographic Signal for Each Polarization Component]
In step S301, the signal processing unit 144 sets an index i of a position yi to one. In steps S302 and S303, the signal processing unit 144 extracts repeated B-scan data (data on m B-scans) of each of the V- and H-polarized components, which are respectively output from the detectors 841 and 842, in the position yi. In steps S304 and S305, the signal processing unit 144 sets an index j of the repeat B-scan to one. In steps S306 and S307, the signal processing unit 144 extracts j-th (1≤j≤m) B-scan data of each of the V- and H-polarized components.
In steps S308 and S309, the image generation unit 147 performs commonly-used reconstruction processing on each of the B-scan data of the V-polarized component and the B-scan data of the H-polarized component extracted by the signal processing unit 144 in steps S306 and S307. Specifically, the image generation unit 147 performs on each of the B-scan data of the V-polarized component and the B-scan data of the H-polarized component the following processing. First, the image generation unit 147 removes fixed pattern noise from the B-scan data. Specifically, a plurality of pieces of A-scan data obtained with the galvano scanner 814 moved to a position such that the measurement light does not enter the eye to be examined 118 is averaged to extract fixed pattern noise, and the extracted fixed pattern noise is subtracted from the input B-scan data to remove the fixed pattern noise. Next, the image generation unit 147 performs desired window function processing to optimize a depth resolution and dynamic range, which are in a tradeoff relationship when Fourier transformed in a finite interval. Thereafter, fast Fourier transformation (FFT) processing is performed to generate tomographic signals. By the above-described processing, two tomographic signals, i.e., a tomographic signal AV of the V-polarized component and a tomographic signal AH of the H-polarized component, and a phase difference ΔΦ between the tomographic signals AV and AH are generated.
In steps S310 and S311, the signal processing unit 144 increments the index j of the repeat B-scan. In steps S312 and S313, the signal processing unit 144 determines whether the index j is larger than the value of m. Specifically, the signal processing unit 144 determines whether the luminance calculation of the B-scan in the position yi is repeated m times. If the signal processing unit 144 determines that the index j is not larger than the value of m (NO in step S312, S313), the processing returns to steps S306 and S307, and the luminance calculation of the repeated B-scans in the same Y-position is repeated. Specifically, the image generation unit 147 obtains a plurality of tomographic signals (tomographic images) of the H- and V-polarized components of the subject which represent tomographic images of substantially the same portion of the subject.
[Generation of Luminance Image]
On the other hand, in steps S312 and S313, if the signal processing unit 144 determines that the index j is larger than the value of m (YES in step S312, S313), the processing proceeds to step S314, and the image generation unit 147 generates a luminance image.
The luminance image is basically the same as a tomographic image in conventional OCT, and a pixel value r of the luminance image is calculated from the tomographic signals AH and AV of the H- and V-polarized components using the following formula (1):
r=√{square root over (AH2+AV2)} (1).
[Generation of Luminance-Averaged Image]
In step S315, the image generation unit 147 performs positioning of the luminance images of the repeated B-scans (m frames) in the yi-position to generate a luminance-averaged image.
Specifically, first, the image generation unit 147 selects any one of the m frames as a template. The template frame can be selected by calculating correlations of all combinations and calculating a sum of correlation coefficients for each frame to select a frame with the largest sum as the template frame. Next, the image generation unit 147 compares each frame with the template to calculate position gap amounts (δX, δY, δθ). Specifically, the image generation unit 147 calculates a normalized cross-correlation (NCC), which is an index indicating similarity, while changing the position and angle of the template image, and determines as the position gap amount a difference between image positions at the maximum NCC value.
In the present exemplary embodiment, the index indicating the similarity can be changed to any one of various measures which indicate the similarity between features of the template and the image in the frame.
Next, the image generation unit 147 applies positional correction to the m−1 frames other than the template according to the position gap amounts (δX, δY, δθ) to register the m frames.
The registered luminance images ri(1≤i≤m) are averaged using the following formula (2) to generate a luminance-averaged image
[Generation of Fibrotic Tissue Image]
In step S316, the image generation unit 147 obtains images of a high-birefringence region and a polarization-maintained region from the tomographic image data of the H- and V-polarized components, and generates a fibrotic tissue image from the two images.
(Retardation Image)
The image of the high-birefringence region is obtained as follows. First, the image generation unit 147 generates a retardation image from the tomographic image data of the H- and V-polarized components. A value δ of a pixel of the retardation image is a quantified phase difference between the V- and H-polarized components in the position of the pixel included in the tomographic image and is calculated from the tomographic signals AH and AV using the following formula (3):
The calculation of the formula (3) is performed for each B-scan image to obtain a retardation image (also referred to as “tomographic image indicating a polarization phase difference”). By generation of the retardation image, it becomes possible to identify a layer with birefringence. Especially a structure that cannot be discriminated from a luminance tomographic image, such as fibrotic tissue, becomes obtainable.
Next, the high-birefringence region is obtained. A boundary of the region to be obtained is determined by, for example, specifying pixels with a larger amount of change in the retardation value in the depth direction than a predetermined threshold value. The amount of change in the depth direction can be calculated from a difference between the retardation values of two adjacent pixels of a pixel or from a gradient of the linear fitting of the retardation values of a plurality of adjacent pixels. Alternatively, the amount of change can be calculated from a gradient of a tangent line at a pixel which is obtained by differentiating the polynomial fitting of the retardation values of a plurality of adjacent pixels. Alternatively, the amount of change can be calculated by performing processing using a Sobel filter or Prewitt filter. In this way, the image of the high-birefringence region is obtained.
(Degree of Polarization Uniformity (DOPU) Image)
The image of the polarization-maintained region is obtained as follows. The image generation unit 147 calculates a Stokes vector S of each pixel from the obtained tomographic signals AH and AV and the phase difference ΔΦ between the tomographic signals AH and AV using the following formula (4):
The phase difference ΔΦ is calculated using ΔΦ=ΦV−ΦH, where ΦV and ΦH are phases of a signal obtained in the calculation of two tomographic images. Next, a window of about 70 μm in the main scanning direction of the measurement light and about 18 μm in the depth direction is set to each B-scan image, and elements of the Stokes vectors calculated for the respective pixels in each window using the formula (4) are averaged. Then, a DOPU in the window is calculated using the following formula (5):
DOPU=√{square root over (Qm2+Um2+Vm2)} (5).
In the formula (5), Qm, Um, and Vm are mean values of elements Q, U, and V of the Stokes vectors in the window. The above-described processing is performed with respect to every window in the B-scan image to generate a DOPU image (also referred to as “tomographic image which indicates a degree of polarization uniformity”). The DOPU is a numerical value which indicates polarization uniformity. In a region in which polarization is maintained, the numerical value of DOPU is close to one. On the other hand, in a region in which polarization is cancelled and not maintained, the numerical value of DOPU is smaller than one. In the inner structure of the retina, the retinal pigment epithelium has the property of cancelling the polarization state. Thus, a portion of the DOPU image that corresponds to the retinal pigment epithelium has a smaller value than the values of other regions. It becomes possible to obtain a structure that cannot be discriminated in the luminance tomographic image, especially the retinal pigment epithelium which becomes discontinuous due to a lesion. Next, threshold value processing (e.g., threshold value 0.75) is performed on the DOPU image to separate the polarization-maintained region and the polarization-cancelled region. In this way, the image of the polarization-maintained region is obtained. Examples of the polarization property image include the DOPU image based on the polarization cancellation information as well as an image based on the retardation information described above (retardation image), an image based on orientation information, and an image based on birefringence information. The images or information are part of three-dimensional polarization sensitive tomographic information.
(Fibrotic Tissue Image)
From the image of the high-birefringence region and the image of the polarization-maintained region obtained by the image generation unit 147 using the above-described method, the map generation unit 148 obtains, as an example of a lesion region, a region which has high birefringence and in which polarization is maintained, thereby generating a fibrotic tissue image f. In the fibrotic tissue image f, a pixel value set to the region which has high birefringence and in which polarization is maintained is different from pixel values set to other regions. For example, in a case in which the fibrotic tissue image f is an 8-bit unsigned image, a pixel value of 0 can be assigned to a region that is not fibrotic, and pixel values corresponding to respective fibrosis levels can be assigned to other regions. The fibrosis levels can be determined according to the degree of birefringence. While the method of obtaining the fibrotic tissue image using the tomographic image data of the H- and V-polarized components of any one frame among the tomographic image data of the polarization components of the m frames obtained by the repeated B-scans is described in the present exemplary embodiment, it is also possible to generate the fibrotic tissue image f using a tomographic image of polarization components of averaged m frames.
[Generation (Segmentation) of Layer Information]
In step S317, the map generation unit 148 generates (segments) retinal layer information from the luminance-averaged image
The retina is known to include ten layers. In the present exemplary embodiment, some of the layers are combined to segment the layers into the following six layers as illustrated in
Next, the following describes a method of obtaining layer information l based on the luminance-averaged image
The map generation unit 148 applies a median filter and a Sobel filter to the tomographic image that is a processing target obtained from the luminance-averaged image
Further, in the segmentation of Layer 5, the map generation unit 148 determines the boundary of RPE using the obtained DOPU image. RPE has the property of cancelling the polarization state due to melanocytes included in RPE, so a portion of the DOPU image that corresponds to RPE has a smaller value than other regions. Specifically, the DOPU image is a visual representation of the property derived from the structure of RPE, so even if RPE is deformed due to a disease, RPE can still be imaged more reliably than the cases of using changes in luminance alone.
Further, the map generation unit 148 calculates the thickness of RPE from the determined boundaries of RPE, maps the calculated thickness, and stores the map as an RPE thickness map in an internal memory (not illustrated). FIG. 4B illustrates an example of the RPE thickness map, and the value of a pixel included in the map corresponds to a thickness of RPE. In the example, a region GA in a center of the map corresponds to a portion the thickness of which is decreased to zero because RPE is atrophied due to atrophic age-related macular degeneration.
[Generation of Blood Vessel Image (Motion Contrast)]
In step S318, the image generation unit 147 generates a blood vessel image from the luminance image.
First, the image generation unit 147 calculates a motion contrast. In the present exemplary embodiment, a variance value of signal intensity (luminance) is calculated for each pixel of the same position from the luminance images of the m frames output by the signal processing unit 144 in step S314, and the calculated variance values are determined as the motion contrast. Specifically, the image generation unit 147 calculates the motion contrast using corresponding pixel data of the plurality of pieces of calculated tomographic image data. Besides the variance value, any one of a standard deviation, difference value, decorrelation value, and correlation value can be used. Further, instead of the signal intensity, the phase can be used. Further, there are various methods for calculating the motion contrast, and as to the type of a feature amount of the motion contrast in the present exemplary embodiment, any index which indicates a change in luminance value of a pixel of the same Y position in the plurality of B-scan images is applicable. Further, the motion contrast can be calculated using a coefficient of variation normalized by a mean value of the same pixel in the respective frames instead of the variance value calculated for each pixel of the same position from the luminance images of the tomographic images of the m frames. The obtained motion contrast data is a part of three-dimensional motion contrast information described below.
Next, the image generation unit 147 performs motion contrast threshold value processing. The image generation unit 147 sets to zero the value of the motion contrast corresponding to a region with a luminance that is equal to or lower than the threshold value. By the threshold value processing, the motion contrast originating from a change in luminance by random noise is removed to reduce noise, and the blood vessels and other portions can be discriminated. The threshold value can be determined using the layer information obtained in step S317. The higher the threshold value is, the smaller the amount of noise becomes, but sensitivity of the motion contrast detection decreased. Thus, since the focus is on new blood vessels relating to RPE in the present exemplary embodiment, the noise threshold value of the motion contrast in the vicinity of RPE or choroid can be lowered. Further, smoothing processing can be performed before or after the threshold value processing. By the above-described processing, a blood vessel image b is generated.
[Generation of Volume Data]
In step S319, the signal processing unit 144 increments the index i of the position yi. In step S320, the signal processing unit 144 determines whether the index i is larger than the value of n. Specifically, whether the predetermined processing is performed at n places in all the y-positions is determined. If the signal processing unit 144 determines that the index i is not larger than the value of n (NO in step S320), the processing returns to steps S302 and S303. On the other hand, if the signal processing unit 144 determines that the index i is larger than the value of n (YES in step S320), the processing proceeds to step S321. After the above-described steps, a three-dimensional image is generated in the next step. Specifically, in step S321, the image generation unit 147 generates a three-dimensional luminance-averaged image
Further, in step S325, the image generation unit 147 records the positions of blood vessels which extend across Layer 5 based on the three-dimensional blood vessel image b3D and the three-dimensional layer information l3D. Further, the image generation unit 147 also records the positions of blood vessels near Layer 6 to Layer 5.
<Display Method>
In step S326, the three-dimensional luminance-averaged image
In a display region 51 in
Further, as illustrated in
For example, when the blood vessel image and the fibrotic tissue are selected to be displayed, the blood vessel image and the fibrotic tissue are displayed in the display region 51 or 61, and the locations and distributions of the blood vessels and the fibrotic tissue can be identified visually. An example is illustrated in
Further, as to a display target region, an image and information on a layer to be focused can be displayed alone in the display region 51 or 61 by selecting the layer to be displayed in the checkbox 55 or 65. In this way, in the case where the target disease is AMD, especially Layers 4 to 6 can be set as the display target, and new blood vessels, which are considered to emerge in the choroid and grow toward the inner layer of the retina, can be targeted and displayed.
Further, in the checkbox 52 or 62, highlighting and displaying the blood vessels near the layers can be selected. An example is illustrated in
Further, the display unit 146 has a function of reading and displaying a previously measured and obtained data set of the same patient. A data set can be read and displayed when the examiner selects an examination number or an examination date. In the examples illustrated in
Further, in the display region 56 or 66, a graph based on the previous data set is displayed, with the vertical axis representing the number and volume of fibrotic tissues calculated in step S323 and the horizontal axis representing the time. The volume of fibrotic tissues can be selected from the volume of total fibrotic tissues included in an examination region and the volume of fibrotic tissues included in an examiner-selected region (e.g., volume of fibrotic tissue in selected layer). In
Further,
The images displayed on the display region 71 are images combined with the blood vessel image using the pixel values of the RPE thickness map as transmittance. In a region with the RPE thickness of zero, only the blood vessel image is displayed. Further, a slider 78 is for adjusting the relationship between the pixel values of the RPE thickness map and the transmittance. In
Further, a target range (layer) of the blood vessel image displayed on the map generation unit 148 can be determined automatically from an image representing polarization characteristics of an eye to be examined. For example, the map generation unit 148 analyzes the fibrotic tissue image and the DOPU image, detects whether there is a fibrotic region and RPE atrophy region in the retina, and automatically checks the checkbox 55, 65, or 75 according to the detection result. In the region detection, a region with a higher pixel value of the fibrotic tissue image than a preset threshold value and a region with the RPE thickness equal to or smaller than a threshold value can be detected. The threshold values can be predetermined by analyzing a plurality of cases of related diseases.
The map generation unit 148 determines Layers 4, 5, and 6 as the display regions of the blood vessel image if a fibrosis region exists in the eye to be examined, or determines Layer 6 as the display region of the blood vessel image if RPE atrophy exists in the eye to be examined. Then, the map generation unit 148 automatically checks the checkboxes simultaneously with superimposing and displaying an image selected to be displayed at that point with the blood vessel image in the layer. Alternatively, the fibrotic tissue image can be selected if a fibrosis region exits, and the RPE thickness map can be selected if RPE atrophy is detected. In this way, the most relevant blood vessel region with respect to a lesion is automatically selected, and this significantly simplifies examiner operations. The relationship between the lesion and the layer to be targeted can be stored in advance in the map generation unit 148 such that the relationship can be changed. In this way, an appropriate layer can be selected according to pathological understanding.
As described above, according to the present exemplary embodiment, the image of the polarization characteristics of the eye to be examined and OCT angiography image are simultaneously presented so that the lesion is effectively displayed to enable efficient diagnosis.
Other Embodiments
Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)), a flash memory device, a memory card, and the like.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2016-091607, filed Apr. 28, 2016, which is hereby incorporated by reference herein in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2016-091607 | Apr 2016 | JP | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 8208996 | Meyer | Jun 2012 | B2 |
| 8857988 | Sharma | Oct 2014 | B2 |
| 9545196 | Abramoff | Jan 2017 | B2 |
| 9700206 | An | Jul 2017 | B2 |
| 9713424 | Spaide | Jul 2017 | B2 |
| 9826900 | Heeren | Nov 2017 | B2 |
| 9955863 | Frisken | May 2018 | B2 |
| 10022047 | Yamashita | Jul 2018 | B2 |
| 10136806 | Sumiya | Nov 2018 | B2 |
| 10152807 | Kuno | Dec 2018 | B2 |
| 20110129133 | Ramos | Jun 2011 | A1 |
| 20140221827 | Motaghiannezam | Aug 2014 | A1 |
| 20160227999 | An | Aug 2016 | A1 |
| Number | Date | Country |
|---|---|---|
| 2008-513067 | May 2008 | JP |
| 2008-154704 | Jul 2008 | JP |
| 2013-542840 | Nov 2013 | JP |
| 2014-083285 | May 2014 | JP |
| 2014-110884 | Jun 2014 | JP |
| 2016-010657 | Jan 2016 | JP |
| 2017006179 | Jan 2017 | JP |
| WO-2016187675 | Dec 2016 | WO |
| Entry |
|---|
| Jeff Fingler et al.; “Mobility and transverse flow visualization using phase variance contrast with spectral domain optical coherence tomography;” Oct. 1, 2007 / Optics Express, vol. 15, No. 20, pp. 12637-12653. |
| Stefan Zotter et al.; “Measuring retinal nerve fiber layer birefringence, retardation, and thickness using wide-field, high-speed polarization sensitive spectral domain OCT;” Investigative Ophthalmology & Visual Science, Jan. 2013, vol. 54, No. 1, pp. 72-84. |
| Number | Date | Country | |
|---|---|---|---|
| 20170316567 A1 | Nov 2017 | US |
