The present invention relates to an image processing apparatus and an image processing method, which are to be used for ophthalmic diagnosis and treatment.
The inspection of an eye has been widely conducted for the purpose of diagnosing and treating lifestyle-related diseases and diseases that are leading causes of blindness in early stages. As an ophthalmic apparatus to be used for the inspection of the eye, there is a scanning laser ophthalmoscope (SLO) using a principle of a confocal laser microscope. The scanning laser ophthalmoscope is an apparatus configured to perform raster scanning on a fundus of an eye with laser light that is measuring light to obtain a planar image of the fundus based on the intensity of return light of the measuring light, and the image is obtained with high resolution at high speed. Further, in the scanning laser ophthalmoscope, the planar image is generated by detecting only light having passed through an aperture portion (pinhole) out of the return light. This allows only return light at a particular depth position to be imaged, and an image having a contrast higher than that of a fundus camera or the like to be acquired.
Such an apparatus configured to photograph a planar image is hereinafter referred to as “SLO apparatus”, and the planar image is hereinafter referred to as “SLO image”.
In recent years, in the SLO apparatus, it has become possible to acquire an SLO image of a retina with improved lateral resolution by increasing a beam diameter of measuring light. However, along with the increase in the beam diameter of the measuring light, an S/N ratio and the resolution of an SLO image of a retina decrease due to an aberration of an eye to be inspected when the SLO image is acquired. The decreases in the resolution are handled by measuring an aberration of an eye to be inspected by a wavefront sensor in real time, and by correcting aberrations of measuring light and return light thereof generated in the eye to be inspected by a wavefront correction device. An adaptive optics SLO apparatus including an adaptive optics system such as the wavefront correction device has been developed to enable the acquisition of an SLO image having a high lateral resolution.
The SLO image obtained by the adaptive optics SLO apparatus can be acquired as a moving image. Therefore, for example, in order to observe hemodynamics non-invasively, the SLO image is used for measurement of the moving speed of blood corpuscles in a capillary vessel and the like through extraction of a retinal vessel from each frame. Further, in order to evaluate a relation with a visual function through use of the SLO image, a density distribution and arrangement of photoreceptor cells P are also measured through detection of the photoreceptor cells P.
In a case of observing the photoreceptor cells P in such an SLO image, a focus position is set to the vicinity of an outer layer of the retina (for example, layer boundary B5 in
However, in a confocal image obtained by imaging the inner layer of the retina, a noise signal is strong due to the influence of light reflected from a nerve fiber layer, and hence it is difficult to observe a blood vessel wall and detect a wall boundary in some cases. In view of the foregoing, in recent years, a method involving obtaining scattering light by changing the diameter, shape, and position of a pinhole arranged in front of a photo-receiving unit and observing a nonconfocal image thus obtained has come to be used (Non Patent Literature 1). In the nonconfocal image, a focus depth is large, and hence an object having irregularities in a depth direction, such as a blood vessel, can be observed easily. Further, light reflected from the nerve fiber layer is not easily received directly, and hence noise can be reduced.
Meanwhile, a retinal artery is an arteriole having a blood vessel diameter of from about 10 μm to about 100 μm, and a wall of the retinal artery is formed of an intima, a media, and an adventitia. Further, the media is formed of smooth muscle cells, and travels along a circumferential direction of the blood vessel in a coil shape. Against a backdrop of hypertension or the like, when pressure exerted on the wall of the retinal artery increases, a smooth muscle contracts to increase a wall thickness. At this point in time, when blood pressure is lowered through administration of an antihypertensive agent, the shape of the wall of the retinal artery returns to an original shape. However, when the hypertension remains untreated for a long period, the smooth muscle cell that forms the media undergoes necrosis, and fibrous hypertrophy of the media and the adventitia occurs to increase the wall thickness. At this point in time, an organic (irreversible) dysfunction has already occurred in the wall of the retinal artery, which necessitates continuous treatment so as to prevent an arteriole dysfunction from becoming worse.
Hitherto, a technology for acquiring the nonconfocal image of the retinal vessel through use of the adaptive optics SLO apparatus and visualizing the retinal vessel wall cells is disclosed in Non Patent Literature 1. In addition, a technology for semiautomatically extracting a retinal vessel wall boundary from an image of an adaptive optics fundus camera through use of a variable shape model is disclosed in Non Patent Literature 2.
The presence or absence and degree of an organic change in the arteriole need to be estimated in the body of a person suffering hypertension, diabetes, or the like. Therefore, it is desired to simply and accurately measure shapes and distributions relating to the walls, membranes, and cells of the retinal artery being an only tissue that can be observed directly among the arterioles of the entire body.
In this connection, a high-resolution image relating to the wall of the retinal artery is acquired through use of an SLO apparatus to which an adaptive optics technology is applied, to thereby allow the observation of the wall of the retinal artery. Further, in a position (Pr1 in
However, when such measurement is performed in a position (Pr2 in
In the technology disclosed in Non Patent Literature 1, the retinal vessel wall, the membrane boundary, and the wall cells are visualized from an AO-SLO image having a nonconfocal image acquisition function based on pinhole control, and the membrane thickness and a cell density are manually measured. However, a technology for automatically measuring the wall thickness and membrane thickness of the retinal vessel and the density of cells that form the wall is not disclosed. Thus, the technology disclosed in Non Patent Literature 1 does not solve the above-mentioned problem.
In the technology disclosed in Non Patent Literature 2, the retinal vessel wall boundary is detected from the image of the adaptive optics fundus camera through the use of the variable shape model, and the wall thickness of the retinal artery is semiautomatically measured. However, a venous wall, or membranes or cells that form an arterial wall and a venous wall cannot be visualized from the image of the adaptive optics fundus camera. That is, a technology for measuring the wall thickness of a vein, the membrane thickness of the artery or the vein, or the distribution of cells that form the blood vessel wall is not disclosed even in Non Patent Literature 2.
Accordingly, there is a demand for a technology for automatically and accurately measuring the wall thickness, the membrane thickness, and the like from the image obtained by visualizing the blood vessel wall of the eye and the membranes and cells that form the blood vessel wall.
NPL 1: Chui et al.; “Imaging of Vascular Wall Fine Structure in the Human Retina Using Adaptive Optics Scanning Laser Ophthalmoscopy”, IOVS, Vol. 54, No. 10, pp. 7115-7124, 2013.
NPL 2: Koch et al.; “Morphometric analysis of small arteries in the human retina using adaptive optics imaging: relationship with blood pressure and focal vascular changes”, Journal of Hypertension, Vol. 32, No. 4, pp. 890-898, 2014.
The present invention has been made in view of the above-mentioned problems, and has an object to accurately measure thicknesses of membranes that form a blood vessel wall of an eye.
In order to attain the object of the present invention, according to one embodiment of the present invention, there is provided an image processing apparatus, including:
an image acquiring unit configured to acquire an image of an eye;
a vessel feature acquiring unit configured to acquire membrane candidate points that form a wall of a blood vessel based on the acquired image;
a cell identifying unit configured to identify a cell that forms the wall of the blood vessel based on the membrane candidate points; and
a measuring position acquiring unit configured to identify a measuring position regarding the wall of the blood vessel based on a position of the identified cell.
Further, according to one embodiment of the present invention, there is provided an image processing method, including:
an image acquiring step of acquiring an image of an eye;
a vessel feature acquiring step of acquiring membrane candidate points that form a wall of a blood vessel based on the acquired image;
a cell identifying step of identifying a cell that forms the wall of the blood vessel based on the membrane candidate points; and
a measuring position identifying step of identifying a measuring position of the blood vessel based on a position of the identified cell.
According to the present invention, it is possible to accurately measure the thicknesses of the membranes that form the blood vessel wall of the eye.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
Now, an image processing apparatus and an image processing method according to an exemplary embodiment of the present invention are described in detail with reference to the accompanying drawings. Note that, the following embodiments are not intended to limit the present invention defined in the appended claims, and not all combinations of features described in the embodiments are essential to solving means of the present invention.
An image processing apparatus according to a first embodiment of the present invention uses an image obtained by imaging a retinal vessel wall through use of an SLO apparatus configured to simultaneously acquire a confocal image and a nonconfocal image. An extreme value of a brightness profile is detected from the image along travel of the wall. Then, cells that form the blood vessel wall are detected based on the obtained extreme value, and a distribution thereof is automatically measured.
Specifically, the retinal vessel wall is imaged through use of the SLO apparatus configured to simultaneously acquire a confocal image and a nonconfocal image. A center line of a retinal vessel (hereinafter referred to also as “blood vessel center line”) is acquired from the obtained nonconfocal image by morphology filter processing. A membrane candidate region that forms the retinal vessel wall is further acquired based on the blood vessel center line. Then, a brightness profile along the travel of a blood vessel wall is generated based on the membrane candidate region. A brightness value within the brightness profile is subjected to a Fourier transform. After a high frequency component is removed from the image that has been subjected to the Fourier transform, a peak position within the brightness profile is detected as the position of the cells. In the following, a case where a membrane thickness is measured by automatically identifying a measuring position of the membrane thickness based on a relative distance between the cells calculated in respective positions along the travel direction or travel line of the blood vessel wall is described.
The SLO image acquiring apparatus 20 is an apparatus configured to acquire a wide field angle image Dl of an eye and a confocal image Dc and a nonconfocal image Dn that are high magnification images. The SLO image acquiring apparatus 20 transmits the wide field angle image Dl, the confocal image Dc, the nonconfocal image Dn, and information on fixation target positions Fl and Fcn used at a time of image acquisition thereof to the image processing apparatus 10 and the data server 40. Note that, the SLO image acquiring apparatus 20 functions as an image acquiring unit configured to acquire the image of the eye in this embodiment.
The pulse data acquiring apparatus 50 is an apparatus configured to acquire biosignal data (pulse data) that changes autonomously, and is formed of, for example, a sphygmograph or an electrocardiograph. The pulse data acquiring apparatus 50 acquires pulse data Pi simultaneously with the acquisition of the wide field angle image Dl, the confocal image Dc, and the nonconfocal image Dn in response to an operation performed by an operator (not shown). The obtained pulse data Pi is transmitted to the image processing apparatus 10 and the data server 40. Note that, the pulse data acquiring apparatus 50 may be directly connected to the SLO image acquiring apparatus 20.
Note that, when the respective images are acquired in different image-acquiring positions, a plurality of images are respectively represented by, for example, Dli, Dcj, and Dnk. That is, i, j, and k are variables each representing an image-acquiring position number, and are set as i=1, 2, . . . , and imax, j=1, 2, . . . , and jmax, and k=1, 2, . . . , and kmax. Further, when the confocal images Dc (nonconfocal images Dn) are acquired with different magnifications, the images are represented by Dc1m, Dc2o, . . . (Dn1m, Dn2o, . . . ) in descending order of the magnification. Further, Dc1m (Dn1m) is represented by a high magnification confocal (nonconfocal) image, and Dc2o, . . . (Dn2o, . . . ) is represented by a medium magnification confocal (nonconfocal) image.
The SLO image acquiring apparatus 20 transmits the wide field angle image Dl, the confocal image Dc, the nonconfocal image Dn, the fixation target positions Fl and Fcn used at the time of the image acquisition, the pulse data Pi, and the like to the data server 40. The data server 40 stores those pieces of information along with image features of the eye output by the image processing apparatus 10. The fixation target positions Fl and Fcn are fixation target positions used at the time of the image acquisition, and it is preferred that other image-acquiring conditions be also stored along with those fixation target positions. Examples of the image features include features regarding the retinal vessel, the retinal vessel wall, and the cells that form the blood vessel wall. Further, in response to a request made by the image processing apparatus 10, the wide field angle image Dl, the confocal image Dc, the nonconfocal image Dn, the pulse data Pi, and the image features of the eye are transmitted to the image processing apparatus 10.
Next, a functional configuration of the image processing apparatus 10 according to this embodiment is described with reference to
Next, the SLO image acquiring apparatus 20 to which adaptive optics used in this embodiment is applied is described with reference to
Measuring light emitted from the SLD 201 serving as a light source passes through an optical path in which the respective optical members are arranged and a crystalline lens OL of an eye E to be inspected to reach a fundus Er of the eye E to be inspected. The measuring light reflected by the fundus Er of the eye follows the optical path backward as return light. A part of return light is split toward the Shack-Hartmann wavefront sensor 206 by the second beam splitter 203. The other part of the return light is further split by the first beam splitter 202 to be guided to the photosensor 211.
The Shack-Hartmann wavefront sensor 206 is a device for measuring an aberration of the eye, and has a CCD 208 connected to a lens array 207. The split part of the return light is transmitted through the lens array 207 as incident light. The incident light transmitted through the lens array 207 appears as a group of bright spots on the CCD 208, and a wavefront aberration of the return light is measured based on a positional deviation of the projected bright spots.
The adaptive optics system 204 drives an aberration correction device to correct the aberration based on the wavefront aberration measured by the Shack-Hartmann wavefront sensor 206. The aberration correction device is formed of a shape variable mirror or a spatial light phase modulator. The return light subjected to aberration correction and split by the first beam splitter 202 passes through the focus lens 209 and the aperture portion 210 to be received by the photosensor 211.
The scan position of the measuring light on the fundus Er of the eye can be controlled by moving the X-Y scanning mirror 205. By the control of the X-Y scanning mirror 205, the operator acquires data on an image acquisition target region specified in advance at a specified frame rate by a specified number of frames. The data is transmitted to the image forming portion 212, and subjected to the correction of an image distortion ascribable to variations in scanning speed and the correction of the brightness value, and image data (moving image or still image) is thus formed. The output portion 213 outputs the image data formed by the image forming portion 212 to the image processing apparatus 10 or the like.
In this case, in the SLO image acquiring apparatus 20, the part of the aperture portion 210 and the photosensor 211 illustrated in
Now, the light shielding portion 210-1 is described with reference to the
The light split by being reflected by the reflection region 210-1-1 of the light shielding portion 210-1 enters the photosensor 211-1. The light that has passed through the transmission regions 210-1-2 and 210-1-3 of the light shielding portion 210-1 is further split by a two-split prism 210-2 arranged on the imaging surface. Light beams obtained after the splitting enter the photosensors 211-2 and 211-3, respectively, as illustrated in
A voltage signal obtained by each of the photosensors is converted into a digital value by an AD board included in the image forming portion 212, and then converted into a two-dimensional image. An image generated based on the light having entered the photosensor 211-1 becomes a confocal image focused within a particular narrow range. Further, an image generated based on the light input to the photosensors 211-2 and 211-3 becomes a nonconfocal image focused within a wide range.
Note that, a method of splitting the return light for extracting a nonconfocal signal is not limited thereto. For example, as illustrated in
In this embodiment, there are two kinds of nonconfocal signals, and hence one is represented by Dnr in the sense of an R-channel image, while the other is represented by Dnl in the sense of an L-channel image.
The expression “nonconfocal image Dn” represents both the R-channel image Dnr and the L-channel image Dnl.
Note that, the SLO image acquiring apparatus 20 according to this embodiment may also be instructed to increase a swing angle of the X-Y scanning mirror 205 serving as a scanning optical system in the configuration of
Note that, in the following, the image having a magnification lower than high magnification images Dc and Dn and having the lowest magnification among the images acquired by the image acquiring portion 110 is referred to as the wide field angle image Dl (Dlc and Dln). Therefore, the wide field angle image Dl may be an SLO image to which the adaptive optics is applied, or may be a mere SLO image. Note that, a confocal wide field angle image and a nonconfocal wide field angle image are represented by Dlc and Dln, respectively, when distinguished from each other.
Next, a hardware configuration of the image processing apparatus 10 according to this embodiment is described with reference to
The functions of the respective blocks that form the image processing apparatus 10 are described in association with a specific execution procedure of the image processing apparatus 10 illustrated in the flowchart of
The image acquiring portion 110 requests the SLO image acquiring apparatus 20 to acquire a low magnification image and a high magnification image. The low magnification image corresponds to the wide field angle image Dl as illustrated in
In response to the acquisition request, the SLO image acquiring apparatus 20 acquires the wide field angle image Dl, the confocal image Dcj, the nonconfocal images Dnrk and Dnlk, corresponding attribute data, and the fixation target positions Fl and Fcn. After the acquisition, those pieces of data are transmitted to the image acquiring portion 110. The image acquiring portion 110 receives the data such as the wide field angle image Dl, the confocal image Dcj, the nonconfocal images Dnrk and Dnlk, the fixation target positions Fl and Fcn from the SLO image acquiring apparatus 20 through the LAN 30, and stores those pieces of data into the memory portion 120.
Further, the pulse data acquiring portion 113 requests the pulse data acquiring apparatus 50 to acquire the pulse data Pi relating to a biosignal. In this embodiment, a sphygmograph is used as the pulse data acquiring apparatus, and the pulse wave data Pi is acquired as the pulse data from a lobulus auriculae (ear lobe) of a subject. Here, the pulse wave data Pi is expressed by a point sequence having one axis indicating an acquisition time and the other axis indicating a pulse wave signal value measured by the sphygmograph. The pulse data acquiring apparatus 50 acquires and transmits the corresponding pulse data Pi in response to the acquisition request. The pulse data acquiring portion 113 receives the pulse data Pi from the pulse data acquiring apparatus 50 through the LAN 30. The pulse data acquiring portion 113 stores the received pulse data Pi into the memory portion 120.
Based on the pulse data Pi acquired by the pulse data acquiring apparatus 50, the confocal data acquiring portion 111 or the nonconfocal data acquiring portion 112 starts acquiring an image. Cases conceivable as modes of the image acquisition include a case where the image acquisition is started in synchronization with a given phase of the pulse data Pi and a case where the acquisition of the pulse data Pi and the image acquisition are simultaneously started immediately after the image acquisition request. In this embodiment, the acquisition of the pulse data Pi and the image acquisition are started immediately after the image acquisition request.
Pieces of pulse data Pi on the respective images are acquired from the pulse data acquiring portion 113, and extreme values of the respective pieces of the pulse data Pi are detected to calculate a heart beat cycle and a relative cardiac cycle. Note that, the relative cardiac cycle is a relative value expressed by a floating-point number ranging from 0 to 1 when the heart beat cycle is set to 1.
Now, examples of the confocal image Dc and the nonconfocal image Dnr obtained when the retinal vessel is imaged are illustrated in
Note that, as the nonconfocal image, any one of an addition-average image Dnr+l (
Note that, the acquisition position of the high magnification image is not limited thereto, and the image in an arbitrary acquisition position may be used. For example, a case of using an image acquired in a macula portion or an image acquired along a retinal vessel arcade is also included in one embodiment of the present invention.
The position alignment portion 131 serving as a position alignment unit performs inter-frame position alignment of the acquired images. Subsequently, the position alignment portion 131 determines an exceptional frame based on the brightness value and noise of each frame and a displacement amount with respect to a reference frame. Specifically, first, the inter-frame position alignment is performed for the wide field angle image Dl and the confocal image Dc. After that, a parameter value of the inter-frame position alignment is also applied to each of the nonconfocal images Dnr and Dnl.
Specifically, the inter-frame position alignment is executed by the position alignment portion 131 with the following procedure.
(i) The position alignment portion 131 first sets the reference frame as the reference of the position alignment. In this embodiment, the frame having the smallest frame number is set as the reference frame. Note that, a method of setting the reference frame is not limited thereto, and an arbitrary setting method may be used.
(ii) The position alignment portion 131 performs rough association of positions between frames (rough position alignment). An arbitrary position alignment method can be used therefor, but in this embodiment, a correlation coefficient is used as an inter-image similarity evaluation function, and affine transformation is used as a coordinate transformation method, to thereby perform the rough position alignment.
(iii) The position alignment portion 131 performs fine position alignment based on data on a correspondence relationship of the rough positions between the frames. In that case, in this embodiment, the fine position alignment between the frames is performed for a moving image obtained by being subjected to the rough position alignment in the stage (ii) through use of the free form deformation (FFD) method that is a kind of non-rigid position alignment method.
Note that, a method for the fine position alignment is not limited thereto, and an arbitrary position alignment method may be used. Further, in this embodiment, a position alignment parameter obtained by performing the inter-frame position alignment of the confocal image Dc is also used as a parameter for the inter-frame position alignment of the nonconfocal image Dn. However, an execution order or the like of the position alignment is not limited thereto. For example, a case of using a position alignment parameter obtained by performing the inter-frame position alignment of the nonconfocal image Dn as a parameter for the inter-frame position alignment of the confocal image Dc is also included in one embodiment of the present invention. In this case, it is preferred that the nonconfocal image Dn include not only Dnr and Dnl described above but also an image obtained by performing arithmetic operation processing for Dnr and Dnl.
Subsequently, the position alignment portion 131 performs the position alignment of the wide field angle image Dl and the high magnification confocal image Dcj (so-called merging of images), and obtains the relative position of the confocal image Dcj on the wide field angle image Dl. In this embodiment, the merging processing is performed through use of superimposed images of the respective moving images. In addition, the merging processing may be performed through use of, for example, the reference frames of the respective moving images. The position alignment portion 131 acquires the fixation target position Fcn used at the time of the image acquisition of the confocal image Dcj from the memory portion 120, and sets the fixation target position Fcn as an initial search point of the position alignment parameter for the position alignment of the wide field angle image Dl and the confocal image Dcj. From then on, the wide field angle image Dl and the confocal image Dcj are subjected to the position alignment while a combination of the parameter values is changed.
The combination of the position alignment parameter values having the highest similarity between the wide field angle image Dl and the confocal image Dcj is determined as the relative position of the confocal image Dcj on the wide field angle image Dl. Note that, the position alignment method is not limited thereto, and an arbitrary position alignment method may be used.
Further, when the image having a medium magnification is acquired in Step S510, the position alignment is performed in ascending order of the magnification from the image having the lowest magnification. For example, when the high magnification confocal image Dc1m and the medium magnification confocal image Dc2o are acquired, it is preferred that the position alignment be first performed between the wide field angle image Dl and the medium magnification image Dc2o. In this case, it is preferred that the above-mentioned position alignment be followed by the position alignment between the medium magnification image Dc2o and the high magnification image Dc1m.
In addition, an image merging parameter value determined for the wide field angle image Dl and the confocal image Dcj is also applied to the merging of the nonconfocal images (Dnrk and Dnlk). Therefore, the relative positions of the high magnification nonconfocal images Dnrk and Dnlk on the wide field angle image Dl are respectively determined.
The vessel feature acquiring portion 132 that functions as a vessel feature acquiring unit and the cell identifying portion 133 that functions as a cell identifying unit identify cells that form the blood vessel wall with the following procedure. That is, the cell identifying unit identifies the cells that form the blood vessel wall based on membrane candidate points that form an arbitrary wall within the blood vessel acquired by the vessel feature acquiring portion 132.
(i) A smoothing process is performed for the nonconfocal image having undergone the inter-frame position alignment in Step S520.
(ii) A morphology filter is applied to detect a retinal artery center line. In each position on the artery center line, the brightness profile on a line segment orthogonal to the artery center line is generated. Then, in regard to the brightness profile, local maximum values are detected at three points from the center of the line segment toward each of the left side and the right side, and are set as candidates for an intima, a media, and an adventitia of the blood vessel wall in the stated order from the position closest to the blood vessel center line. However, it is assumed that the membrane candidate points are not acquired from the brightness profile when the number of detected local maximum points is smaller than three. In addition, membrane candidate points for the media are interpolated along the travel direction of the wall, to thereby generate a curved line along the travel of a blood vessel wall.
(iii) The brightness profile is generated along the curved line generated in the stage (ii). The brightness profile is subjected to a Fourier transform, and then a low-pass filter is applied to a frequency domain, to thereby remove high frequency noise.
(iv) The local maximum values are detected on the brightness profile generated along the travel of the blood vessel wall, which is generated in the stage (iii), to identify positions of the cells that form the blood vessel wall. That is, the cells are identified based on the brightness profile generated along the sequence of the acquired membrane candidate points. The brightness profile can also be generated along a curved line parallel with the blood vessel center line within a blood vessel wall region.
Note that, a specific cell identification process is described in detail with reference to Step S710 to Step S740 illustrated in the flowchart of
The measuring portion 135 calculates a relative distance between the cells based on the positions of the cells that form the blood vessel wall identified in Step S530, and identifies the measuring position of the membrane thickness based on the relative distance. A membrane thickness of the media, a compound membrane thickness of the media and the adventitia, and a wall thickness are measured in the identified measuring position.
A specific measuring process is described in detail with reference to Step S750 to Step S770 illustrated in the flowchart of
The display control portion 136 displays the acquired images, the detected positions of the cells that form the blood vessel wall, and measurement results (density of the cells that form the blood vessel wall, membrane thickness, and wall thickness) on the monitor 305. In this embodiment, the following items (i) to (iv) are displayed. That is,
(i) a nonconfocal moving image (I1 in
an image processed by selecting and superimposing a frame corresponding to a particular phase of a pulse wave (I2 in
an image obtained by extracting the lumen of the blood vessel (13 in
(ii) a map of the detected positions of the cells that form the wall,
(iii) graphs for showing the cell density, the wall thickness, and the membrane thickness measured along the travel of the blood vessel wall (G1 in
(iv) a map for showing the distribution (cell density and area of the cells) of the cells that form the blood vessel wall calculated for each small area (
The instruction acquiring portion 140 acquires from the outside an instruction as to whether or not to store the images acquired in Step S510 and the data on the measurement result obtained in Step S540, that is, the values of the positions of the cells that form the blood vessel wall, the membrane thickness, the wall thickness, the density of the cells that form the blood vessel wall, and the like within the nonconfocal image Dnk, in the data server 40. The instruction is input by the operator through, for example, the keyboard 306 and the mouse 307. When the storing is instructed, the processing advances to Step S570, and when the storing is not instructed, the processing advances to Step S580.
The image processing portion 130 transmits an inspection date/time, information for identifying the eye to be inspected, and the images and the data on the measurement result, which are determined to be stored in Step S560, to the data server 40 in association with one another.
The instruction acquiring portion 140 acquires from the outside an instruction as to whether or not to complete the processing relating to the high magnification nonconfocal image Dnk performed by the image processing apparatus 10. The instruction is input by the operator through the keyboard 306 and the mouse 307. When the instruction to complete the processing is acquired, the processing is brought to an end. Meanwhile, when the instruction to continue the processing is acquired, the processing returns to Step S510 to perform the processing for the next eye to be inspected (or reprocessing for the same eye to be inspected).
Further, the processing executed in Step S530 is described in detail with reference to the flowchart illustrated in
In order to identify the cells that form the blood vessel wall, the cell identifying portion 133 first performs an edge preserving smoothing process for the nonconfocal image. An arbitrary known edge preserving smoothing process is applicable, but in this embodiment, a median value filter is applied to the nonconfocal images Dnr+Dnl.
The morphology filter is applied to the smoothed image generated by the cell identifying portion 133 in Step 5710 to detect the retinal artery center line. In this embodiment, a top-hat filter is applied to detect a high brightness region having a narrow width, which corresponds to blood vessel wall reflection. Further, the high brightness region is subjected to a thinning process to detect the blood vessel center line. Note that, a method of detecting the blood vessel center line is not limited thereto, and an arbitrary known detection method may be used.
Subsequently, the cell identifying portion 133 generates a brightness profile Cr shown in
Note that, a method of acquiring the membrane candidate point sequence is not limited thereto, and an arbitrary known acquisition method may be used. For example, two curved lines parallel with the blood vessel center line are respectively arranged on a blood vessel lumen side and a nerve fiber side as a variable shape model. The model may be deformed so as to match a blood vessel wall boundary by minimizing an evaluation function value regarding the shape and the brightness value on the point sequence that forms the model, and the detected blood vessel wall boundary may be acquired as the membrane candidate point sequence.
The cell identifying portion 133 generates a curved line through the interpolation of the membrane candidate point sequence generated in Step S720, and generates a brightness profile shown in
Subsequently, the high frequency component is removed in order to remove a peak component other than the cells that form the wall (noise or light reflected from a fundus tissue other than the cells that form the wall) from the profile. In this embodiment, the frequency is transformed through use of a Fourier transform, and a low-pass filter is applied to cut a signal value of the higher frequency component. The filtered signal is returned to a spatial domain by being subjected to an inverse Fourier transform, to generate a corrected brightness profile with the high frequency components removed therefrom.
The cell identifying portion 133 detects the local maximum values (Lmm1, Lmm2, and Lmm3 in
Further, the processing executed in Step S540 is described in detail with reference to the flowchart illustrated in
The measuring position identifying portion 134 calculates a conformance degree of the measuring position. In this embodiment, based on the cell positions identified in Step S740, a relative distance Ph obtained when a distance between the cell positions is set as 1 is calculated in the respective positions on a curved line obtained by interpolating the cell positions. This corresponds to a relative phase value obtained when an interval between the center positions of cells distributed at regulating intervals is set as 1 cycle. Specifically, assuming that the center of a cell is 0, the edge of the cell is 0.5, and the center of the adjacent cell is 1. Such a relative distance Ph between the cell positions is calculated for all the membranes whose cells have been detected.
Subsequently, a conformance degree Cf of the measuring position is calculated based on the above-mentioned relative distance Ph. In this embodiment, the conformance degree Cf is calculated as follows.
Cf=|((relative distance Ph between cells)−0.5)|×2.0
Note that, the conformance degree of the measuring position is not limited to the above-mentioned expression for Cf, and an arbitrary expression may be used for the calculation as long as an evaluation value becomes higher in a position closer to the center of a cell and becomes lower in a position closer to the end of the cell.
Note that, in a case of measuring the compound membrane thickness (for example, compound membrane thickness of the media and the adventitia or blood vessel wall thickness), a sum of values obtained by weighting the conformance degrees Cf based on a cell size ratio is calculated as the conformance degree of the measuring position. In this case, a value determined by an arbitrary known method may be used as the cell size ratio. In this embodiment, a value set in advance based on the kind of membrane is used as the cell size ratio. For example, the cell size ratio can be set as (cell of the intima):(cell of the media):(cell of the adventitia)=1:3:1.
The measuring position identifying portion 134 identifies the measuring position based on the conformance degree Cf of the measuring position calculated in Step S750. That is, in this embodiment, the measuring position identifying portion 134 functions as a measuring position acquiring unit configured to identify the measuring position regarding the membrane or the wall of the blood vessel based on the positions of identified cells.
In this embodiment, the membrane thickness is measured by selecting a plurality of positions in which the conformance degree Cf is maximum in each membrane as indicated by the white dashed lines in
Note that, in the case of measuring the compound membrane thickness (for example, compound membrane thickness of the media and the adventitia or blood vessel wall thickness) (
(i) The conformance degree Cf of the measuring position is calculated for each kind of membrane.
(ii) A plurality of positions in which the sum of the values obtained by weighting the conformance degrees Cf of the measuring positions for the respective membranes calculated in the step (i) based on the cell size ratio falls within a predetermined range are identified as the measuring positions.
For example, the cell size ratio is (cell of the intima):(cell of the media):(cell of the adventitia)=1:3:1, and hence the conformance degrees of the measuring positions used for the measurement of the compound membrane thickness of the media and the adventitia can be calculated as follows:
ω1·Cfmω2·Cfa=0.6·Cfm+0.2·Cfa
where Cfm represents the conformance degree of the measuring position for the media, and Cfa represents the conformance degree of the measuring position for the adventitia. In
As described above, the measuring position identifying portion 134 identifies or determines the measuring position based on the distance between predetermined positions within the cells, in this embodiment, between the centers of the cells, identified in at least one of a plurality of membranes that form a blood vessel wall. Note that, the predetermined position may be allowed to be appropriately acquired from an image being displayed or the like.
The measuring portion 135 measures the respective membrane thicknesses of the blood vessel, the compound membrane thickness obtained by summing up the thicknesses of the plurality of membranes, and the wall thicknesses of the blood vessel formed of the plurality of membranes, as measurement values regarding the wall of the blood vessel in the measuring position identified in Step S760. Note that, those measurement items may be at least one of those exemplified above.
Specifically, the mean values, standard deviations, maximum values, and minimum values are respectively calculated for the membrane thickness of the media, the compound membrane thickness of the media and the adventitia, and the wall thickness in the measuring positions identified in Step S760. Those index values are calculated not only as statistics for the entire image, but also in units of a blood vessel branch, units of one side within the blood vessel branch (right side or left side in terms of the vessel travel direction), or units of a small region.
Note that, indices regarding the thicknesses of the membranes that form the blood vessel wall are not limited thereto, and the index values may be calculated by subjecting the values of the membrane thicknesses calculated for the plurality of membranes to an arithmetic operation. For example, the following methods (a) or (b) may be exemplified.
(a) (Compound membrane thickness of the media and the adventitia)/(membrane thickness of the intima)
That is, the compound membrane thickness of the media and the adventitia that easily alter or undergo hypertrophy is standardized with the density of the cells in the intima that relatively hardly change.
(b) A ratio of the membrane thickness (of the same kind) between the left side wall and the right side wall in terms of the vessel travel direction is set as an index.
The wall cells travel in a coil shape, and when a membrane thickness abnormality occurs, the membrane thickness abnormality is considered to be liable to occur on both sides. Therefore, the ratio of the membrane thickness is used as the index of reliability regarding the measurement values of the membrane thickness.
That is, in the measurement of the wall thickness or the like, it is preferred to calculate a specification value based on the membrane thicknesses measured for different membranes of the blood vessel or a new index value obtained by subjecting the index values regarding the membrane thicknesses to an arithmetic operation. Further, such a specification value and an index value can also be used for, for example, determination as to appropriateness of the calculation of the thickness of the actual blood vessel wall.
Note that, as illustrated in
In view of the foregoing, when the cell positions have been detected in a plurality of kinds of membranes and when a cell interval in at least one kind of membrane falls within a predetermined range, it is preferred to identify the measuring position through use of only the conformance degree of the measuring position calculated for the membrane having an appropriate cell interval. When the cell interval is not appropriate in any kind of membrane, it is preferred to identify the measuring position at predetermined intervals along the travel of the blood vessel wall.
According to the above-mentioned configuration, the image processing apparatus 10 performs the following processing for the image acquired by imaging the wall of the retinal artery through the use of the SLO apparatus configured to simultaneously acquire the confocal image and the nonconfocal image. That is, after detecting the cells that form the retinal vessel wall, the image processing apparatus 10 measures the membrane thickness by identifying the measuring position of the membrane thickness based on the relative distance between the cells calculated in the respective positions along the travel of the blood vessel wall.
With this configuration, the thicknesses of the membranes that form the blood vessel wall of the eye can be accurately measured.
The description of the above-mentioned embodiment is directed to the case where the image acquiring portion 110 includes both the confocal data acquiring portion 111 and the nonconfocal data acquiring portion 112. However, the image acquiring portion 110 does not necessarily include the confocal data acquiring portion 111 as long as the configuration allows the acquisition of at least two kinds of non-confocal data.
Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2015-062511, filed Mar. 25, 2015, which is hereby incorporated by reference herein in its entirety.
Number | Date | Country | Kind |
---|---|---|---|
2015-062511 | Mar 2015 | JP | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/JP2016/060285 | 3/23/2016 | WO | 00 |