Patent Application
20150181185
The present invention relates an image processing device for endoscopic diagnosis and a medical endoscope device provided with the image processing device, and particularly to a device having the function of obtaining and displaying information on a lesion portion based on color information of a color endoscopic image.
In general, a lesion portion shows a color different from a color of a normal mucous membrane tissue. Thanks to enhancement of the performance of a color endoscope device, identification of a lesion portion of which color is slightly different from a color of a normal tissue has become possible. However, in order to become able to precisely identify a lesion portion from a normal tissue based on a slight difference in color on an endoscopic image, an operator needs to be trained under expert's instruction for a long period of time. Furthermore, even for a practiced operator, it is not easy to identify a lesion portion based on a slight difference in color, and therefore careful work was required. For this reason, an electronic endoscope device having the function of executing a color conversion process for highlighting a difference in color with respect to endoscopic image data captured using white light, in order to ease identification of a blood vessel or a lesion portion, is proposed, for example, in Japanese Patent Provisional Publication No. 2009-106424A (hereafter, referred to as patent document 1).
It can be said that an image generated by an electronic endoscope described in patent document 1 is easy to identify a lesion portion and the like in comparison with a normal endoscopic image. However, change in color at a boundary between a normal mucous membrane and a lesion portion and the like is continuous, and a difference in color between a normal mucous membrane and a lesion portion becomes very small depending on the type of disease. For this reason, even when the electronic endoscope device described in patent document 1 is used, the problem that identification of a lesion portion and the like is difficult is not resolved. Furthermore, even when the electronic endoscope device described in the patent document 1 is used, judgment on whether or not it is a lesion portion is eventually left to an operator's technical skill which depends on experience or knowledge of the operator, and therefore it is impossible to obtain a diagnostic result having objectivity and reproducibility (which does not depend on the operator's skill).
According to an embodiment of the invention, there is provided an image processing device, comprising an image obtaining means that obtains color endoscopic image data, a color space conversion means that converts a color space of the color endoscopic image data to an HSI color space based on hue, saturation and intensity or an HSV color space based hue, saturation and a value, a lesion pixel judgment means that judges, for each of pixels constituting a color endoscopic image, whether or not each pixel is a lesion portion based on the hue and the saturation, a lesion pixel counting means that counts a number of lesion pixels which is a number of pixels judged to be the lesion portion; a lesion index calculation means that calculates a lesion index based on the number of lesion pixels, and a lesion index display means that displays the lesion index.
With this configuration, since the judgment on the lesion portion is made in the HSV color space where the difference between the lesion portion and the normal portion clearly appears, is becomes possible to make an accurate judgment on the lesion portion and the normal portion. Further, through display of the lesion index, the objective and quantitative information on the lesion portion photographed on the endoscopic image is obtained, and therefore it becomes possible to conduct a diagnosis with high accuracy and reproducibility.
In the above described image processing device, the image processing device may further comprise an effective pixel judgment means that judges, for each of the pixels constituting the color endoscopic image, whether or not each pixel is effective by judging whether or not a pixel value is within a predetermined range; and an effective pixel counting means that counts a number of effective pixels which is a number of pixels judged to be effective by the effective pixel judgment means, and wherein the lesion pixel judgment means may judge whether each pixel is the lesion portion for the pixel judged to be effective, and the lesion index display means may calculate the lesion index as a ration of the number of lesion pixels to the number of effective pixels.
With this configuration, by excluding the non-effective pixel (e.g., a pixel with a low accuracy of a pixel value, such as a case of an extremely high pixel value or an extremely low pixel value) from a judgment target, the more accurate identification of the lesion portion can be achieved.
In the above described image processing device, the lesion pixel judgment means may judge whether or not each pixel constituting the color endoscopic image is the lesion portion, for each of a plurality of types of lesions, the lesion pixel counting means may count the number of lesion pixels, for each of the plurality of types of lesions, the lesion index calculation means may calculate the lesion index, for each of the plurality of types of lesions, and the lesion index display means may display the lesion index of at least one of the plurality of types of lesions.
With this configuration, the lesion index is displayed for each of the types of lesions, am operator is able to obtain detailed information on the state of a patient, and therefore it becomes possible to conduct the more accurate diagnosis.
In the above described image processing device, the image processing device may further comprise a total lesion index calculation means that calculates a total lesion index which is a sum of the lesion indexes of the plurality of types of lesions, and wherein the lesion index display means may display the total lesion index.
With this configuration, since the operator is able to obtain the objective and total information concerning diseases showing various types of lesions whose diagnosis is difficult, it becomes possible to easily conduct suitable diagnosis.
In the above described image processing device, the lesion pixel counting means may count the number of lesion pixels as a number of pixels judged to be pixels of the lesion portion of one of the plurality of types of lesions.
With this configuration, it becomes possible to provide the accurate information concerning a state of a disease, even for a disease with a plurality of types of lesions for which identification by image analysis is difficult.
In the above described image processing device, the image processing device may further comprise a tone enhancement means that executes a tone enhancement process applying a non-linear gain to a pixel value so that a dynamic range near a boundary of a region of the pixel value for which the pixel is judged to be the lesion portion is broadened, and wherein the lesion pixel judgment means may make a judgment based on the pixel value after the tone enhancement process.
With this configuration, a dynamic range near a boundary (a threshold value) of judgment on a lesion portion is widened, it becomes possible to conduct the more precise judgment on a lesion portion.
In the above described image processing device, the color endoscopic image data may be converted into the HSI color space or the HSV color space after being subjected to the tone enhancement process in the RGB color space.
In the above described image processing device, the tone enhancement means may execute the tone enhancement process while applying different gain curves to intensity values of respective primary colors of R, G and B.
With this configuration, it becomes possible to conduct the more accurate judgment on a lesion portion by the tone enhancement process having a high degree of freedom.
In the above described image processing device, the image processing device may further comprise an overlay means that executes an overlay process which changes a color of the pixel judged to be the lesion portion by the lesion pixel judgment means, and an overlay image display means that displays an overlay image obtained by subjecting the color endoscopic image to the overlay process.
With this configuration, a portion judged to be a lesion portion on the overlay image is explicitly indicated, it becomes possible to prevent overlooking of a lesion portion and to conduct the more accurate diagnosis.
In the above described image processing device, the overlay image display means may display concurrently the color endoscopic image and the overlay image.
With this configuration, by comparison with the overlay image, identification of a lesion portion photographed on the color endoscopic image is eased.
In the above described image processing device, the overlay means may add a predetermined value corresponding to a type of lesion to a pixel value of a pixel judged to be the pixel of the lesion portion.
With this configuration, lesion portions of different types are displayed according to their colors, it becomes possible to recognize intuitively and accurately the more detailed information on a state of a disease.
In the above described image processing device, the image processing device may further comprise a image capturing position obtaining means that obtains information on a position at which the color endoscopic image is captured, and a report display means that displays, using at least one of a table, a graph and a schematic diagram, a relationship between a plurality of positions at which the color endoscopic image is captured and the lesion index of the color endoscopic image captured at the positions.
With this configuration, it becomes possible to easily recognize the accurate whole aspect of a disease.
In the above described image processing device, the report display means may execute printing and displaying.
In the above described image processing device, the lesion may include an inflammation and an ulcer in an inflammatory bowel disease.
With this configuration, since the lesion index becomes a suitable index of the severity of an inflammatory disease, the more appropriate diagnosis on the severity of an inflammatory disease is enabled.
Furthermore, according to an embodiment of the invention, there is provided an endoscope device, comprising the above described image processing device, and an endoscope that generates the color endoscopic image data and outputs the color endoscopic image data to the image processing device.
As described above according to the embodiment of the invention, it becomes possible to conduct the accurate identification between a lesion portion and a normal portion.
In the following, an embodiment of the invention is explained in detail with reference to the accompanying drawings.
The processor 200 for an electronic endoscope includes a system controller 202 and a timing controller 206. The system controller 202 executes various programs stored in a memory 204 and totally controls the entire electronic endoscope device 1. Further, the system controller 202 changes various settings of the electronic endoscope device 1 in accordance with an instruction from a user (an operator or an assistant) input to an operation panel 208. The timing controller 206 outputs clock pulses for adjusting operation timings of each part to various circuits in the electronic endoscope device 1.
Further, the processor 200 for an electronic endoscope includes a light source 230 which supplies illumination light being a white light beam to an LCB (Light Carrying Bundle) 102 of the electronic scope 100. The light source 230 includes a lamp 232, a lamp power supply 234, a condenser lens 236 and a dimmer control device 240. The lamp 232 is a high luminance lamp which emits illumination light while being supplied drive power from the lamp power supply 234, and, for example, a Xenon lamp, a metal-halide lamp, a mercury lamp or a halogen lamp is used. The illumination light emitted by the lamp 232 is converged by the condenser lens 236, and is guided to the LCB 102 via the dimmer control device 240.
The dimmer control device 240 is a device which adjusts the light amount of the illumination light guided to the LCB 102 based on control by the system controller 202, and includes an aperture stop 242, a motor 243 and a driver 244. The driver 244 generates a driving current for driving the motor 243, and supplies the driving current to the motor 243. The aperture stop 242 is driven by the motor 243, and changes the aperture through which the illumination light proceeds, and thereby controls the light amount of the illumination light passing through the aperture.
The illumination light guided to the LCB 102 from an entrance face propagates through the inside of the LCB 102, is emitted from an exit face of the LCB 102 disposed at a tip part of the electronic scope 100, and illuminates a subject via a light distribution lens 104. Light reflected from the subject forms an optical image on a light-receiving surface of a solid state image pickup device 108 via an objective lens 106.
The solid state image pickup device 108 is, for example, a single-chip color CCD (Charge Coupled Device) image sensor having various filters, such as, an IR (Infra Red) cut filer 108a and a bayer array color filter 108b, disposed on the light-receiving surface thereof, and generates an image-capturing signal of each of three primary colors responsive to the optical image converged on the light receiving surface. The generated image-capturing signal is amplified by a driver signal processing circuit 112 provided in a connection part of the electronic scope 100, and then is converted into an image signal composed of a luminance signal Y and color difference signals Cb and Cr. The image signal converted into the luminance signal Y and the color difference signals Cb and Cr is further converted into a digital signal and then is transmitted to an image processing unit 220 of the processor 200 for an electronic endoscope. Further, the driver signal processing circuit 112 reads out unique information unique to the electronic scope 100 by accessing the memory 114. The unique information of the electronic scope 100 recorded in the memory 114 includes, for example, the number of pixels and the sensitivity of the solid state image pickup device 108, operable frame rates and the like. The driver signal processing circuit 112 outputs the unique information read from the memory 114 to the system controller 202.
The system controller 202 executes various calculations based on the unique information of the electronic scope 100, and generates a control signal. Using the generated control signal, the system controller 202 controls the operations and timings of the various circuits in the processor 200 for an electronic endoscope so that processes suitable for the electronic scope 100 connected to the processor 200 for an electronic endoscope are executed.
The timing controller 206 supplies clock pulses to the driver signal processing circuit 112 and the image processing unit 220 in accordance with timing control by the system controller 202. The driver signal processing circuit 112 drives and controls the solid state image pickup device 108 at timings in synchronization with a frame rate of video processed by the processor 200 for an electronic endoscope side, in accordance with the clock pulses supplied from the timing controller 206.
The image processing unit 220 of the processor 200 for an electronic endoscope generates a video signal for monitor representation of an endoscopic image or the like based on the image signal transmitted from the driver signal processing circuit 112 of the electronic scope 100, under control of the system controller 202, and outputs the video signal to the monitor 300. The operator observes or treats, for example, an inside of a digestive tube, while checking the endoscopic image displayed on the monitor 300. Further, the image processing unit 220 generates a report output signal (a print signal) for printing out a report screen, which is described later, based on the image signal, and outputs the report output signal to the printer 400.
The processor 200 for an electronic endoscope is connected to a server 600 via a NIC (Network Interface Card) 210 and a network 500. The processor 200 for an electronic endoscope obtains, from the server 600, information concerning endoscopic examination (e.g., electronic medical record of a patient or information on an operator) and displays the information on the monitor 300 or the operation panel 208. Further, the processor 200 for an electronic endoscope transmits endoscopic examination results (endoscopic image data, examination conditions, image analysis results which are described later, an operator's opinion or the like) to the server 600 to cause the server 600 to store the endoscopic examination results.
Further, the electronic endoscope device 1 includes the function of recording still images of a plurality of endoscopic images in association with information on captured portions (i.e., positions (insertion lengths) of the tip part of the electronic scope 100 at the time of image capturing). On an outer circumferential surface of an insertion unit 130 of the electronic scope 100, a plurality of optical sensors 132 are provided at constant intervals (e.g., intervals of 5 centimeters) in the length direction. The optical sensor 132 is a light-receiving device, such as a photodiode, and detects external light (room lighting of a room in which the endoscopic examination is being performed). The optical sensors 130 provided on a part of the insertion unit 130 inserted into the digestive tube do not detect the external light, and the optical sensors 132 provided on a part of the insertion unit 130 not inserted into the digestive tube detect the external light. Therefore, by judging that the length of distribution of the optical sensors not detecting the light is the length of the insertion unit 130 inserted into the inside of the digestive tube, it becomes possible to obtain information concerning the position of the tip part of the electronic scope 100 (the insertion length). The optical sensors 132 are connected to the driver signal processing circuit 112, and transmit sensor signals indicating the amounts of the detected light to the driver signal processing circuit 112. The driver signal processing circuit 112 calculates the position (the insertion length) Pos of the tip part of the electronic scope 100 based on the sensor signals from the optical sensors 132.
When an operation for instructing obtaining of a still image is conducted with respect to the operation unit 120 of the electronic scope 100, an operation signal is transmitted from the operation unit 120 to the driver signal processing circuit 112. When the system controller 202 obtains the operation signal for obtaining a still image from the operation unit 120, the system controller 202 transmits an instruction for obtaining a still image to the image processing unit 220 together with the current position information (the insertion length) Pos of the tip part of the electronic scope 100. As a result, in the image processing unit 220, a still image of an endoscopic observation image is recorded while being associated with the position information Pos of the electronic scope 100 at the time of image capturing. Details about a process in which the image processing unit 220 records a still image are described later.
As shown in
The memory area groups Pk and Qk (k=1 to 6) are associated with the position information (the insertion length) Pos of the tip part of the electronic scope 100, respectively. Specifically, the memory area groups P1 and Q1 correspond to a range of the insertion length Pos of a deepest portion (e.g., a portion near a right colic flexure of a transverse colon) in an examination area, and the insertion length Pos becomes shorter as the value of k increases. The memory area groups P6 and Q6 correspond to a range of the insertion length Pos corresponding to an area around a rectum. That is, when still images are obtained while drawing the insertion unit 130 of the electronic scope 100 from the deepest portion of the examination area, the still images are recorded in the memory areas of k=1 to 6 in the order that the still images are obtained. The setting information that defines the correspondence between the position information Pos and the memory are groups Pk, Qk (k=1 to 6) is recorded in the setting information memory area 226c. The memory area groups Pk, Qk in which the image data is recorded is determined depending on the position information Pos (the position of the tip part of the electronic scope 100 at the time of image capturing) of the image data.
The second image processing circuit 228 generates the video signal for monitor representation using the image signal stored in the image memory 224, and outputs the video signal to the monitor 300. Further, the second image processing circuit 228 generates the report output signal (the print signal) using the image signal stored in the image memory 224 and the lesion index (which is described later) stored in the memory 226, and outputs the report output signal to the printer 400.
Next, details about a process executed by the image processing unit 220 are explained.
Next, it is determined whether the mode is set to the image analysis mode (S2). The image analysis mode according to the embodiment of the invention is an operation mode in which color information (in particular, hue and saturation) is analyzed for each pixel of the image data, and whether it is a pixel of a lesion portion is judged based on a predetermined judgment criteria from an analysis result of the color information, and the pixel of the lesion portion is displayed in a discriminating manner. The type of lesion to be judged can be selected depending on content of the examination. In an example explained below, pixels in color regions respectively specific to observation images of an ulcer (a white lesion including a white moss or mucous) which is a lesion of inflammatory bowel disease (IBD) and inflammation (a red coloration lesion including an edema or hemorrhagic) are extracted, and are displayed in a discriminating manner.
The electronic endoscope device 1 according to the embodiment is configured to operate in two operation modes including the image analysis mode and the normal observation mode. The operation mode is changed by a user operation to the operation unit 120 of the electronic scope 100 or the operation panel 208 of the processor 200 for an electronic endoscope. When the operation mode is set to the normal observation mode (S2: No), the process proceeds to S9.
When the image analysis mode is selected (S2: Yes), next the TE process S3 by the tone enhancement (TE) processing unit 222b is executed. The TE process S3 is a process in which, in order to enhance the judgment accuracy of lesion, the gain adjustment for giving a non-linear gain to each of the primary color signals R, G, B is executed, and the dynamic range around a color region (in particular, a boundary portion) specific to the lesion to be judged is substantially widened, and the effective resolution for color representation is enhanced. Specifically, in the TE process S3, a process where a monotonically increasing non-linear gain as shown in
Here, different gain adjustments may be executed for the primary color signals R, G, B, respectively. Furthermore, a process in which the same non-liner gain is given to the primary color signals G, B and a different non-linear gain is given to the primary color signal R may be performed. Further, the three primary signals R, G′, B′ (the tone enhanced image data E) generated by the TE process S3 are output to the image memory 224, and are stored in the tone enhanced image memory areas 0eR, 0eG, 0eB, respectively.
Through the TE process S3, an inflammation portion changes in color to red, an ulcer portion changes in color to white, and a normal portion changes in color to green. Therefore, when the tone enhanced image data E generated by the TE process S3 is displayed on the monitor 300, it becomes possible to visually recognize a lesion portion (an inflammation portion or an ulcer portion) easily relative to the normal observation image data N.
When the TE process S3 is completed, next an effective pixel judgment process S4 by the effective pixel judgment unit 222c is executed for the tone enhanced image data E.
(Expression 1)
int(x,y)=0.3*R′(x,y)+0.59*G′(x,y)+0.11*B′(x,y)
The value of the calculated correction intensity (x, y) is used for a suitable exposure judgment S42. Further, as can be seen from the expression 1, the correction intensity int(x, y) is not obtained as a simple average of the primary color signals R′(x, y), G′(x, y), B′(x, y), but is obtained as a weighted average based on a relative luminous efficiency property of a human (an operator).
Next, the suitability of exposure (whether or not it is an exposure level required for image analysis) is judged based on the correction intensity int(x, y) and the primary color signals R′(x, y), G′(x, y), B′(x, y) of the tone enhanced image data E calculated in the process S41 (S42). In the suitable exposure judgment process S42, it is judged to be a suitable exposure (S42: Yes) when at least one of the following two conditions (the expression 2 and the expression 3) is satisfied. It should be noted that the upper limit of the correction intensity int(x, y) is defined by the expression 2, and the lower limit of each of the primary color signals R′(x, y), G′(x, y), B′(x, y) is defined by the expression 3.
(Expression 2)
int(x,y)<235
(Expression 3)
Max {R′(x,y),G′(y),B′(x,y)}>20
When, for the pixel (x, y), the expressions (2) and (3) are satisfied and it is judged to be the suitable exposure (S42: Yes), the effective pixel judgment unit 222c changes the value of the flag (x, y) corresponding to the pixel (x, y) of the flag table F stored in the flag memory area 226b of the memory 226 to “1” (S43). The flag f (x, y) takes a value of one of 0 to 3. The following is the definition of each of the flag values.
0: Invalid image data
1: No lesion (Normal) or Non-judgment on lesion (image data effective)
2: Lesion A (Inflammation)
3: Lesion B (Ulcer)
When none of the expression 2 and the expression 3 is satisfied and the exposure is judged to be not suitable in the suitable exposure judgment S42 (S42: No), the effective pixel judgment unit 222c changes the value of the flag f(x, y) of the flag table F to “0” (S44).
Next, in the process S45, it is judged whether the process is completed for all the pixels (x, y). Until the process is completed for all the pixels (x, y), the above described processes S41 to S45 are repeated.
When the effective pixel judgment process S4 is completed, next a color space conversion process S5 is executed by the color space conversion unit 222d. The color space conversion process S5 is a process where the tone enhanced image data in the RGB space defined by the RGB three primary colors is converted into image data in HSI (Hue-Saturation-Intensity) space defined by three elements of Hue, Saturation and Intensity. Specifically, in the color space conversion process S5, the primary color signals R′(x, y), G′(x, y), B′(x, y) of each pixel (x, y) of the tone enhanced image data E are converted into hue H(x, y), saturation S(x, y), intensity I(x, y). Since the intensity I(x, y) is not used in the following process in this embodiment, calculation for the intensity I(x, y) is not executed and only the hue H(x, y) and the saturation S(x, y) are calculated so that the processing efficiency is enhanced.
Data of a pixel whose exposure is short or excessive has a low accuracy, and deteriorates the reliability of the analysis result. Therefore, the color space conversion process S5 is executed only for the pixels (x, y) for which the value of the flag f (x, y) is set to “1” (i.e., the pixel (x, y) judged to be suitable exposure in the above described effective pixel judgment process S4).
When the color space conversion process S5 is completed, then a lesion judgment process S6 is executed by the lesion judgment unit 222e. The lesion judgment process S6 according to the embodiment of the invention is a process where, for each pixel (x, y) of the endoscopic image, a predicted state (normal, inflammation, ulcer) of a portion corresponding to the pixel is judged depending on which of regions I to III of
Here, the inflammation judgment process S62 and an ulcer judgment process S64 described later are explained.
In the inflammation judgment process S62, it is judged whether or not each pixel value (H(x, y), S(x, y)) is plotted in the region II of
(Expression 4)
130+δH≦H(x,y)
(Expression 5)
60+δS1≦S(x,y)≦100+δS2
When the pixel value (H(x, y), S(x, y)) is plotted in the region II (S62: Yes), the flag f (x, y) corresponding to the pixel (x, y) is rewritten to “2” (inflammation) (S63), and the process proceeds to the process S66. When the pixel value (H(x, y), S(x, y)) is not plotted in the region II (S62: No), an ulcer judgment process S64 is executed subsequently.
In the ulcer judgment process S64, it is judged whether or not the pixel value (H(x, y), S(x, y)) is plotted in the region III of
(Expression 6)
S(x,y)≦65+δS3
(Expression 7)
S(x,y)≦85+δS4
(Expression 8)
120+δH2≦H(x,y)≦200+δH3
When the pixel value (H(x, y), S(x, y)) is plotted in the region III (S64: Yes), the value of the flag f(x, y) corresponding to the pixel (x, y) is rewritten to “3” (ulcer) (S65), and the process proceeds to process S66. When the pixel value (H(x, y), S(x, y)) is not plotted in the region III (64: No), the pixel is judged to be a normal portion, and the process proceeds directly to the process S66 without rewriting the value “1” of the flag f (x, y).
In the process S66, it is judged whether or not the process is completed for all the pixels (x, y). Until the process is completed for all the pixels (x, y), the above described processes S61 to S66 are repeated.
When the lesion judgment process S6 is completed, next the overlay process S7 according to the embodiment of the invention is executed by the overlay process unit 222f. The overlay process S7 is a process where, for the pixel judged to be a lesion portion such as inflammation or an ulcer by the lesion judgment process S6, the color is changed to have a color tone associated with the lesion (the flag value) so that the pixel can be distinguished from an pixel judged to be normal (not having a lesion). In this embodiment, a process where red color is enhanced (specifically, a red component is increased) for the pixel judged to be an ulcer and yellow color is enhanced (specifically, green and blue components are enhanced) for a pixel judged to be inflammation is executed.
In the process S77, it is judged whether the process is completed for all the pixels (x, y). Until the process is completed for all the pixels (x, y), the above described processes S71 to S77 are repeated.
When the overlay process S7 is completed, the overlay image data S generated by the overlay process S7 is output to the image memory 224, and are stored in the overlay image memory areas 0sR, 0sG, 0sB (k=0 to 6), respectively.
Next, a lesion index calculation process S8 according to the embodiment of the invention is executed by the lesion index calculation unit 228a of the second image processing circuit 228. The lesion index calculation process S8 is a process where a ratio (a lesion index) of the number of pixels judged to have a lesion in the lesion judgment process S6 to the number of effective pixels (the number of pixels judged to be suitable exposure in the effective pixel judgment process S6) of the endoscopic image is calculated.
Next, it is judged whether or not each pixel (x, y) is the effective pixel with reference to the flag table F (S82). When the pixel is not the effective pixel (S82: 0), the process proceeds to S87 without updating the counters. When the pixel is the effective pixel (S82: 1-3), the effective pixel counter Num0 is incremented (S83). Next, it is judged which of the normal, inflammation and ulcer each pixel (x, y) is judged to belong to in accordance with the value of the flag f (x, y) (S84). When the pixel has been judged to be normal (S84: 1), the process proceeds directly to S87. When the pixel has been judged to be inflammation (S84: 2), the inflammation counter NumR is incremented (S85). When the pixel has been judged to be ulcer (S84: 3), the ulcer counter NumW is incremented (S86), and then the process proceeds to S87.
Next, it is judged whether the process is completed for all the pixels (x, y) (S87). Until the process is completed for all the pixels (x, y), the above described processes S81 to S87 are repeated.
When the counting is completed for all the pixels (x, y), next the lesion index IdxR for inflammation is calculated (S88). The lesion index IdxR for inflammation is a ratio of the number of pixels judged o be inflammation to the number of all the effective pixels, and is calculated by IdxR=NumR/Num0.
Next, the lesion index idxW for ulcer is calculated (S89). The lesion index IdxW for ulcer is a ratio of the number of pixels judged to be ulcer to the number of all the effective pixels, and is calculated by IdxW=NumW/Num0.
Next, the total lesion index IdxT is calculated (S90). The total lesion index IdxT is a ratio of the number of pixels judged to be lesion (inflammation or ulcer) to the number of all the effective pixels, and is calculated by IdxT=IdxE+IdxW. Through study by the inventors of the invention, it has been founded that the total lesion index IdxT of an endoscopic image of inflammatory bowel disease and the Mayo score which indicates the severity of inflammatory bowel disease strongly correlate to each other, and the total lesion index IdxT can be used as an appropriate index for simply judging the severity of the inflammatory bowel disease.
Next, each of the lesion indexes IdxR, IdxW and IdxT is recorded in the memory 226, and the lesion index calculation process S8 terminates.
Next, a judgment S9 (
The next screen display process S11 is a process where the display screen data for displaying on the monitor 300 is generated and is converted into a video signal and is output, and is executed by the display screen generating unit 228b of the second image processing circuit 228. The display screen generating unit 228b is able to generate a plurality of types of display screen data in accordance with control by the system controller 202.
In the screen display process S11, the display screen generating unit 228b reads the real time lesion indexes IdxR, IdxW and IdxR from the memory area group Q0 of the memory 226, and displays them on the lesion index display area 323. In
The operator conducts the endoscopic observation while viewing the analysis mode observation screen 320. Specifically, the operator conducts the endoscopic observation while viewing the normal observation image data N (or the tone enhanced image data E) displayed on the normal image display area 324 and while referring to the overlay image data S displayed on the analysis image display area 325. By carefully observing particularly for a colored portion on the overlay image data S, it becomes possible to conduct accurate diagnosis without overlooking a lesion portion. Furthermore, by conducting observation while referring to objective and concrete numeric values of the lesion indexes IdxR, IdxW and IdxT displayed on the lesion index display area 323, more objective and precise diagnosis can be conducted irrespective of a subjective impression received from the normal observation image data N or the overlay image data S.
Next, it is judged whether an instruction for executing report output according to the embodiment of the invention is received from the system controller 202 (S12). When an instruction for executing report output is issued by a user operation to the operation unit 120 of the electronic scope 100 or the operation panel 208 of the processor 200 for an electronic endoscope, the system controller 202 transmits the execution instruction for the report output process 513 described later to the image processing unit 220. When the execution instruction for the report output process S12 has been issued by the system controller 202 (S12: Yes), subsequently the report output process S13 is executed. When the execution instruction has not been issued (S12: No), the process proceeds to the process S14 without executing the report output process S13.
The report output process S13 is a process where a report output screen 420 in which results of an endoscopic examination are arranged is generated, and they are converted into a report signal (a print signal) and is output to the printer 400, and the report output screen 420 is printed. The report output screen shown in
The large bowel model diagram 421 illustrates positions at which still images have been obtained (observation points), and a plurality of circular symbols 421b respectively indicating the observation points 1 to 6 are arranged on a schematic diagram 421a of a large bowel. The observation points 1 to 6 respectively correspond to the memory area groups P1 to P6 of the image memory 224 and the memory area groups Q1 to Q6 of the memory 226. The lesion index list 422 lists the total lesion indexes IdxT (422c) at the observation points 1 to 6 (422b). The lesion index distribution diagram 423 represents, as a graph, the lesion index list 422. The statistic list 424 lists the statistics (an average, a variance and a standard deviation) of the total lesion index IdxT. The thumbnail image list 425 lists still image thumbnail images 425b of the normal observation image data N (or the tone enhanced image data E) obtained at the observation points 1 to 6 (425a), still image thumbnail image 425c of the overlay image data S and the medical record number 425d assigned to the normal observation image data N.
Through use of the report screen 420, change of an endoscopic image and the lesion index in a longer direction of an intestinal tract can be recognized at a glance, and it becomes possible to appropriately and easily make explanations about the examination results to a patient as well as the diagnosis.
Next, it is judged whether or not the endoscopic observation should continue (S14). The above described processes S1 to S14 are repeated until a user operation for instructing stop of endoscopic examination or stop of operation of the electronic endoscope device 1 is conducted to the operation panel 208 of the processor 200 for an electronic endoscope (S14: No).
The forgoing is the explanation about the embodiment; however, the invention is not limited to the above described embodiment but can be varied within the scope of the technical concept expressed by the claims.
Although, in the above described embodiment, judgment on a lesion is made in the HSI space, judgment on a lesion may be made in the HSV (HSB) space based on Hue, Saturation and Value (or Brightness) in place of HSI space.
In the above described embodiment, the TE process S3 is executed in the RGB space; however, the TE process S3 may be executed in the HSI space after the color space conversion process S5.
In the above described embodiment, the configuration where the information on the position of the tip part of the electronic scope 100 (the image capturing position) is obtained using the optical sensor 132; however, the invention is not limited to such a configuration, and a configuration where the information on the image capturing position is obtained by another means may be employed. For example, a configuration where an optical proximity sensor having a light source and a light-receiving element are provided in the insertion unit 130 of the electronic scope 100 in place of the optical sensor 132 may be employed. In this case, when the optical proximity sensor is inserted in to a digestive tube, light reflected from the inner wall of the digestive tube is detected by the light-receiving element, and proximity is detected. Therefore, in contrast to the above described embodiment, by judging that the distribution length of the optical proximity-sensors whose light-receiving elements are receiving light is the length of the insertion unit 130 being inserted into the digestive tube, the information on the position of the tip part of the electronic scope 100 can be obtained.
A configuration where a moving amount sensor which detects the moving amount (the distance and the direction of movement) by the same principle as that of an optical mouse is provided in the insertion unit 130 of the electronic scope 100 in place of the optical sensor 132 may be employed. In this case, only one moving amount sensor may be provided close to the tip of the insertion unit 130.
It should be noted that, as a light source of the optical proximity-sensor or the moving amount sensor, a light source (LD or LED) of a wavelength within near infrared to a visible light region may be used; however, by using a light source of a red light region having a high degree of reflectivity on a living tissue surface, a high degree of detection accuracy can be obtained.
Furthermore, by obtaining a magnetic resonance image, an X-ray image or an ultrasonic image from the outside of a patient body during endoscopic examination, the position of the tip part of the electronic endoscope may be determined from these images. Furthermore, by calculating the moving amount of the tip part of the endoscope in a digestive tract through image analysis of an endoscopic image, the position of the tip part of the endoscope may be determined.
The above described embodiment is an example in which the invention applied to endoscopic examination for inflammatory diseases; however, as a matter of course, the invention may be applied to endoscopic examination for another disease. It should be noted that since, regarding an inflammatory disease, the ratio of a lesion portion increases as the range and the severity of a lesion portion (an inflammatory portion or an ulcer portion) increases, the total lesion index IdxT becomes a suitable index for the severity. On the other hand, since, regarding a disease where a lesion portion exists locally (e.g., polyp), the angle of view of a lesion portion becomes larger as the tip portion of the electronic scope becomes closer to the lesion portion, the total lesion index IdxT increases. Therefore, the total lesion index IdxT becomes a suitable index indicating whether it is close to a lesion portion, and by making careful observation when the total lesion index IdxT starts to increase, it becomes possible to prevent looking over of a lesion portion.
Furthermore, since, in the above described embodiment, the judgment conditions of the inflammation (region II) and the ulcer (region III) do not overlap with each other as shown in
Although the above described embodiment is configured such that a still image is recorded by a manual operation by an operator, a configuration where the driver signal processing circuit 112 automatically issues an instruction for storing a still image and automatically storing a still image when the tip part of the insertion unit 130 reaches a predetermined still image obtaining position (an observation point) while the insertion unit 130 of the electronic scope 100 is pulled out from the deepest portion of an examination area.
In the above described embodiment, a CCD image sensor is used as the solid state image pickup device 108; however, a configuration where a solid state image pickup device having a different configuration, such as a CMOS (Complementary Metal Oxide Semiconductor), is used may be employed.
In the above described embodiment, the solid state image pickup device 108 having the RGB bayer array color filter 108b is used; however, a configuration having a solid state image pickup device having a complementary color filter of Cy (Cyan), Mg (Magenta), Ye (Yellow) and G (Green) may be employed.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2012-158249 | Jul 2012 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2013/063384 | 5/14/2013 | WO | 00 |
