Patent Grant
11523733
The present invention relates to an endoscope system and a method of operating the endoscope system that illuminate an object with a plurality of kinds of illumination light having different wavelength ranges while switching the plurality of kinds of illumination light and display observation images corresponding to the plurality of kinds of illumination light while switching the observation images.
In recent years, an endoscope system comprising a light source device, an endoscope, and a processor device has been widely used in a medical field. In the endoscope system, an object to be observed is irradiated with illumination light from an endoscope, and the image of the object to be observed is displayed on a monitor on the basis of RGB image signals that are obtained in a case where the image of the object to be observed, which is being illuminated with the illumination light, is picked by an image pickup element of the endoscope.
Further, in recent years, an object to be observed has been illuminated with a plurality of kinds of illumination light having different wavelength ranges to obtain much diagnosis information from the object to be observed. For example, JP2012-213551A (corresponding to US2012/0253158A1) discloses a device that illuminates an object to be observed with wavelength sets formed of narrow-band light having wavelengths of four hundreds nm, five hundreds nm, and six hundreds nm while automatically switching the narrow-band light to allow a user to observe the oxygen saturation of superficial blood vessels, the oxygen saturation of middle-layer blood vessels, and the oxygen saturation of deep blood vessels included in the object to be observed.
In a case where an object to be observed is illuminated with a plurality of kinds of illumination light having different wavelength ranges as described above, observation images corresponding to the plurality of kinds of illumination light are different from each other in appearance. For example, a blue observation image obtained from illumination using blue light is displayed so that superficial blood vessels are emphasized, but a green observation image obtained from illumination using green light is displayed so that medium-deep blood vessels are emphasized. Since both the superficial blood vessels and the medium-deep blood vessels are important in a diagnosis, a user makes a diagnosis while confirming both the blue observation image and the green observation image.
Here, in a case where a user is to frequently confirm a plurality of observation images, a plurality of kinds of illumination light are automatically switched and observation images corresponding to the plurality of kinds of illumination light are displayed while being automatically switched. Accordingly, a user can confirm the respective observation images without a burden. Further, in a case where a plurality of observation images are to be displayed while being automatically switched as described above, the display contents of each observation image need to be set so that specific subject information (for example, a lesion) included in the object to be observed is emphasized and the other subject information (for example, normal mucous membranes) is not emphasized. Accordingly, the display contents of each observation image need to be capable of being set without applying a load to the entire device to emphasize only specific subject information.
An object of the invention is to provide an endoscope system and a method of operating the endoscope system that can set the display contents of each observation image without applying a load to the entire device to emphasize only specific subject information in a case where a plurality of kinds of illumination light are emitted while being switched and observation images corresponding to the plurality of kinds of illumination light are displayed while being switched.
An endoscope system according to an aspect of the invention comprises a plurality of semiconductor light sources, a light source control unit, an image acquisition unit, and a display control unit. The plurality of semiconductor light sources emit light having wavelength ranges different from each other. The light source control unit controls the plurality of semiconductor light sources. The light source control unit performs control to emit a plurality of kinds of illumination light, which include first illumination light having a first light emission ratio and second illumination light having a second light emission ratio different from the first light emission ratio, while switching the plurality of kinds of illumination light according to a specific light emission pattern. The image acquisition unit acquires a plurality of observation images obtained from image pickup of an object to be observed illuminated with each illumination light. The plurality of observation images include a first observation image corresponding to the first illumination light and a second observation image corresponding to the second illumination light. The display control unit performs control to display the plurality of observation images on a display unit while switching the plurality of observation images according to a specific display pattern. The specific light emission pattern is fixed and the specific display pattern is changeable.
It is preferable that the specific light emission pattern is a pattern where the first illumination light is emitted for a time corresponding to the first number of light emission frames, the first illumination light is then switched to the second illumination light, and the second illumination light is emitted for a time corresponding to the second number of light emission frames in a predetermined specific light emission cycle; the specific display pattern is a pattern where the first observation image is displayed on the display unit for a time corresponding to the first number of display frames, the first observation image is then switched to the second observation image, and the second observation image is displayed on the display unit for a time corresponding to the second number of display frames in the specific light emission cycle; and the first number of light emission frames and the second number of light emission frames are fixed, and the first number of display frames and the second number of display frames are changeable.
It is preferable that, in a case where a first display pattern where the first number of display frames is set to be larger than the second number of display frames is set, a first observation image within a second illumination light-emission period, which is displayed on the display unit in a light emission period of the second illumination light, is displayed on the basis of a first observation image within a first illumination light-emission period that is obtained in a light emission period of the first illumination light prior to the light emission period of the second illumination light.
It is preferable that, in a case where a second display pattern where the first number of display frames is set to be smaller than the second number of display frames is set, a second observation image within a first illumination light-emission period, which is displayed on the display unit in a light emission period of the first illumination light, is displayed on the basis of a second observation image within a second illumination light-emission period that is obtained in a light emission period of the second illumination light prior to the light emission period of the first illumination light. It is preferable that, in a case where a second display pattern where the first number of display frames is set to be smaller than the second number of display frames is set, the first observation image is not displayed on the display unit in a light emission period of the first illumination light.
It is preferable that the endoscope system further comprises a display pattern switching unit that is used to switch a first display pattern where the first number of display frames is set to be larger than the second number of display frames and a second display pattern where the first number of display frames is set to be smaller than the second number of display frames. It is preferable that the display unit displays a switching display screen that displays the plurality of observation images while switching the plurality of observation images and a single-image display screen that displays only any one of the plurality of observation images. It is preferable that, in a case where only any one of the first observation image or the second observation image is displayed in the single-image display screen, the number of display frames of the observation image, which is not displayed in the single-image display screen, of the first and second observation images is set to “0”. It is preferable that the display control unit cancels an observation image immediately after switching, which is obtained immediately after switching of each illumination light, not to display the observation image on the display unit.
A method of operating an endoscope system according to another aspect of the invention comprises a light source control step, an image acquisition step, and a display control step. In the light source control step, a light source control unit, which controls a plurality of semiconductor light sources emitting light having wavelength ranges different from each other, performs control to emit a plurality of kinds of illumination light, which include first illumination light having a first light emission ratio and second illumination light having a second light emission ratio different from the first light emission ratio, while switching the plurality of kinds of illumination light according to a specific light emission pattern. In the image acquisition step, an image acquisition unit acquires a plurality of observation images obtained from image pickup of an object to be observed illuminated with each illumination light, and the plurality of observation images include a first observation image corresponding to the first illumination light and a second observation image corresponding to the second illumination light. In the display control step, a display control unit performs control to display the plurality of observation images on a display unit while switching the plurality of observation images according to a specific display pattern. The specific light emission pattern is fixed and the specific display pattern is changeable.
According to the invention, it is possible to set the display contents of each observation image without applying a load to the entire device to emphasize only specific subject information in a case where a plurality of kinds of illumination light are switched and observation images corresponding to the plurality of kinds of illumination light are displayed while being switched.
As shown in
Further, the operation part 12b is provided with a mode changeover SW 13a in addition to the angle knobs 12e. The mode changeover SW 13a is used for an operating for switching a mode to a normal observation mode, a first special observation mode, a second special observation mode, and a multi-observation mode. The normal observation mode is a mode where a normal image is displayed on the monitor 18. The first special observation mode is a mode where a first observation image where superficial blood vessels are emphasized is displayed on the monitor 18. The second special observation mode is a mode where a second observation image where deep blood vessels are emphasized is displayed on the monitor 18. The first special observation mode and the second special observation mode are automatically switched in the multi-observation mode, so that the first and second observation images are displayed on the monitor 18 while being switched according to a specific display pattern.
A foot switch may be used as a mode switching unit, which is used to switch a mode, other than the mode changeover SW 13a. Further, the operation part 12b is provided with a freeze button (not shown) that is used to acquire a static image. In a case where a user detects a portion considered to be effective for diagnosis, the mode changeover SW 13a and the freeze button are operated alternately.
The processor device 16 is electrically connected to the monitor 18 and the user interface unit 19. The monitor 18 outputs and displays image information and the like. The user interface unit 19 includes a keyboard, a mouse, and the like, and receives an input operation, such as function settings. An external recording unit (not shown), which records image information and the like, may be connected to the processor device 16.
As shown in
The light source control unit 21 controls the drive of the LEDs 20a to 20d. The optical path-combination unit 23 combines the optical paths of pieces of light that are emitted from the four color LEDs 20a to 20d and have four colors. The inside of an object to be examined is irradiated with the pieces of light, which are combined by the optical path-combination unit 23, through a light guide 41 inserted into the insertion part 12a and an illumination lens 45.
As shown in
The light source control unit 21 performs control to turn on the V-LED 20a, the B-LED 20b, the G-LED 20c, and the R-LED 20d in all observation modes. Further, the light source control unit 21 controls the respective LEDs 20a to 20d so that normal light of which the light intensity ratios of violet light V, blue light B, green light G, and red light R are Vc:Bc:Gc:Rc is emitted in the normal observation mode. In this specification, a light emission ratio means the light intensity ratio of each semiconductor light source and includes a case where the light intensity ratio is 0 (zero). Accordingly, the light emission ratio includes a case where any one or two of the respective semiconductor light sources are not turned on. For example, even in a case where only one semiconductor light source is turned on and the other three semiconductor light sources are not turned on as in a case where the light intensity ratios of violet light V, blue light B, green light G, and red light R are 1:0:0:0, it is considered that the light source unit has a light emission ratio.
Furthermore, in the first special observation mode, the light source control unit 21 controls the respective LEDs 20a to 20d to emit first illumination light of which the light emission ratios of violet light V, blue light B, green light G, and red light R are Vs1:Bs1:Gs1:Rs1. It is preferable that the first illumination light has a peak in the range of 400 nm to 440 nm. For this purpose, Vs1:Bs1:Gs1:Rs1 of the first illumination light are set so that the light intensity of violet light V is higher than the light intensity of each of blue light B, green light G, and red light R as shown in
Further, since the first illumination light includes a first red-light wavelength range like red light R, the first illumination light can accurately reproduce the color of a mucous membrane. Furthermore, since the first illumination light includes a first blue-light wavelength range and a first green-light wavelength range like violet light V, blue light B, and green light G, the first illumination light can also emphasize various structures, such as glandular structures and unevenness, in addition to the above-mentioned superficial blood vessels.
Moreover, in the second special observation mode, the light source control unit 21 controls the respective LEDs 20a to 20d to emit second illumination light of which the light emission ratios of violet light V, blue light B, green light G, and red light R are Vs2:Bs2:Gs2:Rs2. It is preferable that the intensity ratio of the second illumination light is higher than that of the first illumination light at wavelengths of 460 nm, 540 nm, and 630 nm.
For this purpose, Vs2:Bs2:Gs2:Rs2 of the second illumination light are set so that the amounts of blue light B, green light G, and red light R of the second illumination light are larger than the amounts of blue light B, green light G, and red light R of the first illumination light as shown in
Vs2:Bs2:Gs2:Rs2 are set so that the light intensity of violet light V is lower than the light intensity of each of blue light B, green light G, and red light R (Vs2<Bs2, Gs2, and Rs2). Further, since the second illumination light includes a second red-light wavelength range like red light R, the second illumination light can accurately reproduce the color of a mucous membrane. Furthermore, since the second illumination light includes a second blue-light wavelength range and a second green-light wavelength range like violet light V, blue light B, and green light G, the second illumination light can also emphasize various structures, such as glandular structures and unevenness, in addition to the above-mentioned medium-deep blood vessels.
Further, in a case where a mode is set to the multi-observation mode, the light source control unit 21 performs control to emit the first illumination light and the second illumination light while automatically switching the first illumination light and the second illumination light according to a specific light emission pattern. The specific light emission pattern is a pattern where the first illumination light is emitted for a time corresponding to the first number of light emission frames, the first illumination light is then switched to the second illumination light, and the second illumination light is emitted for a time corresponding to the second number of light emission frames in a predetermined specific light emission cycle. Here, in this embodiment, a light emission time where illumination light is emitted is represented by a value converted into the number of frames that is a unit used for the control of an image pickup sensor 48 (see
The first number of light emission frames and the second number of light emission frames in the above-mentioned specific light emission cycle are fixed. Here, “fixed” means that a specific light emission pattern cannot be changed by a user after the specific light emission pattern, such as the light emission order of the first illumination light and the second illumination light in addition to the first number of light emission frames and the second number of light emission frames, is set at the time of the shipment of the light source device 14 from the factory. For example, a user cannot change the specific light emission pattern in a case where the endoscope 12 is used, that is, in a case where the light source device 14 to which the endoscope 12 is to be connected is used. However, the change of the specific light emission pattern, which is performed by a person authorized to change the specific light emission pattern, such as the administrator of the light source device 14, is not hindered. Further, each of the numbers of light emission frames is set to a period of two or more frames. The reason why each of the numbers of light emission frames is set to a period of two or more frames as described above is that the illumination light of the light source device 14 is immediately switched but the image processing of the processor device 16 has at least two or more frames without being immediately switched.
As shown in
The distal end part 12d of the endoscope 12 is provided with an illumination optical system 30a and an image pickup optical system 30b. The illumination optical system 30a includes an illumination lens 45, and an object to be observed is irradiated with light transmitted from the light guide 41 through the illumination lens 45. The image pickup optical system 30b includes an objective lens 46 and an image pickup sensor 48. Light reflected from the object to be observed is incident on the image pickup sensor 48 through the objective lens 46. Accordingly, the reflected image of the object to be observed is formed on the image pickup sensor 48.
The image pickup sensor 48 is a color image pickup sensor, and picks up the reflected image of an object to be examined and outputs image signals. It is preferable that the image pickup sensor 48 is a charge coupled device (CCD) image pickup sensor, a complementary metal-oxide semiconductor (CMOS) image pickup sensor, or the like. The image pickup sensor 48 used in the invention is a color image pickup sensor that is used to obtain RGB image signals corresponding to three colors of R (red), G (green), and B (blue), that is, a so-called RGB image pickup sensor that comprises R-pixels provided with R-filters, G-pixels provided with G-filters, and B-pixels provided with B-filters.
The image pickup sensor 48 may be a so-called complementary color image pickup sensor, which comprises complementary color filters corresponding to C (cyan), M (magenta), Y (yellow), and G (green), instead of an RGB color image pickup sensor. In a case where a complementary color image pickup sensor is used, image signals corresponding to four colors of C, M, Y, and G are output. Accordingly, the image signals corresponding to four colors of C, M, Y, and G need to be converted into image signals corresponding to three colors of R, G, and B by complementary color-primary color conversion. Further, the image pickup sensor 48 may be a monochrome image pickup sensor that includes no color filter. In this case, since the light source control unit 21 causes blue light B, green light G, and red light R to be emitted in a time-sharing manner, demosaicing needs to be added to the processing of image pickup signals.
The image signals output from the image pickup sensor 48 are transmitted to a CDS/AGC circuit 50. The CDS/AGC circuit 50 performs correlated double sampling (CDS) or auto gain control (AGC) on the image signals that are analog signals. The image signals, which have been transmitted through the CDS/AGC circuit 50, are converted into digital image signals by an analog/digital converter (A/D converter) 52. The digital image signals, which have been subjected to A/D conversion, are input to the processor device 16.
The processor device 16 corresponds to a medical image processing device that processes medical images, such as images obtained by the endoscope 12. The processor device 16 comprises an image acquisition unit 53, a digital signal processor (DSP) 56, a noise removing unit 58, an image processing unit 60, a parameter switching unit 62, a display control unit 66, and a central control unit 68. Digital color image signals output from the endoscope 12 are input to the image acquisition unit 53. The color image signals are RGB image signals formed of R-image signals that are output from the R-pixels of the image pickup sensor 48, G-image signals that are output from the G-pixels of the image pickup sensor 48, and B-image signals that are output from the B-pixels of the image pickup sensor 48.
The DSP 56 performs various kinds of signal processing, such as defect correction processing, offset processing, gain processing, color adjustment processing, gamma conversion processing, and demosaicing processing, on the received image signals. Signals of defective pixels of the image pickup sensor 48 are corrected in the defect correction processing. Dark current components are removed from the RGB image signals having been subjected to the defect correction processing in the offset processing, so that an accurate zero level is set.
The RGB image signals having been subjected to the offset processing are multiplied by a specific gain parameter in the gain processing, so that signal levels are adjusted. The specific gain parameter varies for each observation mode. For example, gain processing for normal light for multiplying image signals, which are obtained from the illumination of the normal light and image pickup, by a gain parameter for normal light as the specific gain parameter is performed in the normal observation mode. Further, gain processing for first illumination light for multiplying RGB image signals (first observation image), which are obtained from the illumination of the first illumination light and image pickup, by a gain parameter for first illumination light as the specific gain parameter is performed in the first special observation mode. Furthermore, gain processing for second illumination light for multiplying RGB image signals (second observation image), which are obtained from the illumination of the second illumination light and image pickup, by a gain parameter for second illumination light as the specific gain parameter is performed in the second special observation mode.
Further, in the multi-observation mode, the gain processing for first illumination light is performed on RGB image signals, which are obtained from the illumination of the first illumination light and image pickup, at the time of the illumination of the first illumination light, and the gain processing for second illumination light is performed on RGB image signals, which are obtained from the illumination of the second illumination light and image pickup, at the time of the illumination of the second illumination light.
After that, brightness or a saturation is adjusted by the gamma conversion processing. The demosaicing processing (also referred to as equalization processing or demosaicing) is performed on the RGB image signals having been subjected to the color adjustment processing, so that signals of colors deficient in each pixel are generated by interpolation. All the pixels are made to have the signals of the respective colors of R, G, and B by this demosaicing processing.
The noise removing unit 58 performs noise removal processing (for example, a moving-average method, a median filtering method, or the like) on the RGB image signals, which have been subjected to gamma correction and the like by the DSP 56, to remove noise from the RGB image signals. The RGB image signals from which noise has been removed are transmitted to the image processing unit 60.
The image processing unit 60 performs various kinds of image processing on the RGB image signals. The various kinds of image processing include image processing that is performed under a condition varying for each observation mode in addition to image processing that is performed under the same condition regardless of an observation mode. The image processing that is performed under a condition varying for each observation mode includes color adjustment processing for improving color reproducibility and structure emphasis processing for emphasizing various structures, such as blood vessels and unevenness. Each of the color adjustment processing and the structure emphasis processing is processing that uses a two-dimensional look up table (LUT), a three-dimensional look up table (LUT), a matrix, or the like. In a case where the color emphasis processing and the structure emphasis processing are to be performed, a color emphasis processing parameter and a structure emphasis processing parameter set for each observation mode are used in the image processing unit 60. The switching of the color emphasis processing parameter or the structure emphasis processing parameter is performed by the parameter switching unit 62.
In a case where a mode is set to the normal observation mode, a parameter to be used in the image processing unit 60 is switched to a color emphasis processing parameter for normal light and a structure emphasis processing parameter for normal light by the parameter switching unit 62. Then, the image processing unit 60 performs color emphasis processing for normal light on the RGB image signals using the color emphasis processing parameter for normal light, and performs structure emphasis processing for normal light on the RGB image signals using the structure emphasis processing parameter for normal light. After that, the RGB image signals having been subjected to the above-mentioned processing are input to the display control unit 66 as the normal image.
In a case where a mode is set to the first special observation mode, a parameter to be used in the image processing unit 60 is switched to a color emphasis processing parameter for first illumination light and a structure emphasis processing parameter for first illumination light by the parameter switching unit 62. Then, the image processing unit 60 performs color emphasis processing for first illumination light on the RGB image signals using the color emphasis processing parameter for first illumination light, and performs structure emphasis processing for first illumination light on the RGB image signals using the structure emphasis processing parameter for first illumination light. After that, the RGB image signals having been subjected to the above-mentioned processing are input to the display control unit 66 as the first observation image.
In a case where a mode is set to the second special observation mode, a parameter to be used in the image processing unit 60 is switched to a color emphasis processing parameter for second illumination light and a structure emphasis processing parameter for second illumination light by the parameter switching unit 62. Then, the image processing unit 60 performs color emphasis processing for second illumination light on the RGB image signals using the color emphasis processing parameter for second illumination light, and performs structure emphasis processing for second illumination light on the RGB image signals using the structure emphasis processing parameter for second illumination light. After that, the RGB image signals having been subjected to the above-mentioned processing are input to the display control unit 66 as the second observation image.
In a case where a mode is set to the multi-observation mode, the image processing unit 60 performs the color emphasis processing for first illumination light and the structure emphasis processing for first illumination light on the RGB image signals at the time of the illumination of the first illumination light. After that, the RGB image signals having been subjected to the above-mentioned processing are input to the display control unit 66 as the first observation image. Further, the image processing unit 60 performs the color emphasis processing for second illumination light and the structure emphasis processing for second illumination light on the RGB image signals at the time of the illumination of the second illumination light. Furthermore, in a case where a mode is set to the multi-observation mode, the image processing unit 60 performs mucous membrane-color-balance processing for setting the colors of normal mucous membranes, which are included in the object to be observed, to the same color between the first observation image and the second observation image. First mucous membrane-color-balance processing is performed on the first observation image, and second mucous membrane-color-balance processing based on the result of the first mucous membrane-color-balance processing is performed on the second observation image. After that, the RGB image signals having been subjected to the above-mentioned processing are input to the display control unit 66 as the second observation image.
B1-image signals, G1-image signals, and R1-image signals included in the first observation image are automatically adjusted in the first mucous membrane-color-balance processing as described in, for example, D1) to D3) to be described below so that the average color of the entire screen has a specific color balance. The first mucous membrane-color-balance processing is performed on the assumption that the color of a mucous membrane is dominant over the object to be observed. Then, the first mucous membrane-color-balance processing is performed, so that B1*-image signals, G1*-image signals, and R1*-image signals having been subjected to the first mucous membrane-color-balance processing are obtained.
B1*-image signal=B1/B1ave D1)
G1*-image signal=G1/G1ave D2)
R1*-image signal=R1/R1ave D3)
Here, B1ave denotes the average pixel value of the B1-image signals (the sum of pixel values of the entire screen (effective pixels)/the number of effective pixels). G1ave denotes the average pixel value of the G1-image signals (the sum of pixel values of the entire screen (effective pixels)/the number of effective pixels). R1ave denotes the average pixel value of the R1-image signals (the sum of pixel values of the entire screen (effective pixels)/the number of effective pixels).
Further, B2-image signals, G2-image signals, and R2-image signals included in the second observation image are automatically adjusted in the second mucous membrane-color-balance processing as described in, for example, E1) to E3) to be described below so that the average color of the entire screen has a specific color balance. B1ave, G1ave, and R1ave calculated in the first mucous membrane-color-balance processing are used in the second mucous membrane-color-balance processing. Then, the second mucous membrane-color-balance processing is performed, so that B2*-image signals, G2*-image signals, and R2*-image signals having been subjected to the second mucous membrane-color-balance processing are obtained.
B2*image signal=B2/B1ave E1)
G2*image signal=G2/G1ave E2)
R2*image signal=R2/R1ave E3)
The display control unit 66 performs control to display the normal image, the first observation image, or the second observation image, which is input from the image processing unit 60, as an image that can be displayed on the monitor 18. In the normal observation mode, the display control unit 66 displays the normal image on the monitor 18. In the first special observation mode, the display control unit 66 displays the first observation image on the monitor 18. In the second special observation mode, the display control unit 66 displays the second observation image on the monitor 18. In the multi-observation mode, the display control unit 66 displays the first observation image or the second observation image on the monitor 18 while switching the first and second observation images according to a specific display pattern. The details of display control in the multi-observation mode will be described later.
The central control unit 68 controls the respective parts of the processor device 16. Further, the central control unit 68 receives information from the endoscope 12 or the light source device 14, and performs the control of the respective parts of the processor device 16 and the control of the endoscope 12 or the light source device 14 on the basis of the received information.
For example, in a case where an observation mode is switched in the endoscope 12, information about the switching of a mode is transmitted to the central control unit 68 as shown in
Further, in a case where the central control unit 68 receives information about the switching of a mode, the central control unit 68 instructs the DSP 56 or the parameter switching unit 62 provided in the processor device 16 to change processing according to the switching of a mode. For example, in a case where a mode is switched to the second special observation mode from the first special observation mode, the DSP 56 switches gain processing to the gain processing for second illumination light from the gain processing for first illumination light by switching a gain parameter to the gain parameter for second illumination light from the gain parameter for first illumination light. Further, the parameter switching unit 62 switches color adjustment processing to color adjustment processing for second illumination light from color adjustment processing for first illumination light by switching a color adjustment processing parameter to a color adjustment processing parameter for second illumination light from a color adjustment processing parameter for first illumination light. Furthermore, the parameter switching unit 62 switches structure emphasis processing to the structure emphasis processing for second illumination light from the structure emphasis processing for first illumination light by switching a structure emphasis processing parameter to the structure emphasis processing parameter for second illumination light from the structure emphasis processing parameter for first illumination light.
The switching of the gain processing, the color adjustment processing, and the like having been described above is often not performed immediately due to a processing situation or the like in the processor device 16. For example, there is a case where two or more frames are required after a mode is switched until gain processing, color adjustment processing, and the like are completely switched to the gain processing, the color adjustment processing, and the like corresponding to the switched mode. Accordingly, it is preferable that the number of frames is set to two or more.
In the multi-observation mode, the display control of a plurality of observation images is performed according to a specific display pattern. The specific display pattern is a pattern where the first observation image is displayed on the monitor 18 for a time corresponding to the first number of display frames, the first observation image is then switched to the second observation image, and the second observation image is displayed on the monitor 18 for a time corresponding to the second number of display frames in the specific light emission cycle. Here, in this embodiment, a time where the observation image is displayed on the monitor 18 is represented by a value converted into the number of frames that is a unit used for the control of the image pickup sensor 48 (see
The first number of display frames or the second number of display frames can be changed by a user, and the change of each of the numbers of display frames is performed in a number-of-display-frame-setting menu 70 shown in
Here, the first number of display frames and the second number of display frames are determined so that a difference in specific subject information (for example, blood vessel portions) included in the first and second observation images can be emphasized. Here, in a case where each of the first and second observation images is displayed for a relatively long time and are switched, there is a case where it is difficult to notice a difference between these images, particularly, a difference between blood vessels. Further, in a case where the first number of display frames or the second number of display frames is reduced so that the first and second observation images are displayed while being switched at a high speed in a short time, the first and second observation images are recognized as not two kinds of observation images but one kind of observation image. Accordingly, it is preferable that each of the first number of display frames and the second number of display frames is set to a predetermined number of frames or more (for example, three or more frames) so that a difference in specific subject information of two kinds of images can be emphasized and the first and second observation images are not recognized as one kind of image.
Furthermore, in a case where the first number of display frames and the second number of display frames are set to be equal to each other so that the first and second observation images are displayed for the same display time while being switched, the first and second observation images may flicker. For this reason, one of the first number of display frames and the second number of display frames is set to be long and the other thereof is set to be short, so that the first and second observation images can be displayed while a difference between the first and second observation images is emphasized. In a case where a third observation image obtained on the basis of third illumination light having a wavelength range between the wavelength of the first illumination light and the wavelength of the second illumination light is displayed while being switched in addition to the first observation image or the second observation image, blood vessels extending to a superficial layer from a medium-deep layer can be made to be three-dimensionally displayed. In this case, for example, it is preferable that the number of display frames of an observation image (for example, the first observation image) serving as a base is set to be large and the third observation image where blood vessels extending to a superficial layer from a medium-deep layer are emphasized and the second observation image where medium-deep blood vessels are emphasized are sequentially displayed by a small number of frames.
The first number of display frames can be changed between, for example, “0” and “10”. In a case where the first number of display frames is to be changed, a user operates the user interface unit 19 to move a slider 72a to the left side or the right side on a slide bar 72 representing the first number of display frames. As a result, the first number of display frames is changed. Further, the second number of display frames can also be changed between “0” and “10” as with the first number of display frames. Even in a case where the second number of display frames is to be changed, a user operates the user interface unit 19 to move a slider 74a to the left side or the right side on a slide bar 74 representing the second number of display frames. As a result, the second number of display frames is changed. In the case of the slide bar 72, the first number of display frames is larger as the slider is closer to the right side. The same applies to the slide bar 74.
However, the total number of display frames, which is the sum of the first number of display frames and the second number of display frames in the specific light emission cycle, is fixed. For example, the total number of display frames is fixed to “10”. Accordingly, in a case where one of the first number of display frames and the second number of display frames is increased, the other thereof is reduced by an increase in one of the first number of display frames and the second number of display frames. For example, in a case where the total number of display frames is set to “10” and the first number of display frames is changed to “8”, the second number of display frames is changed to “2” with a change in the first number of display frames.
Further, in this embodiment, the first number of light emission frames or the second number of light emission frames is not changed to be fixed and only the first number of display frames or the second number of display frames is changed. Here, in a case where not only the first number of display frames or the second number of display frames but also the first number of light emission frames or the second number of light emission frames is to be changed, the light emission timing of the illumination light of the light source device 14 and the processing timing of the image processing of the processor device 16 need to be synchronized with each other. However, since the time required to change the light emission timing of the illumination light and the time required to change the processing timing of the image processing are different from each other, a certain time or more is required to synchronize these times. For this reason, a load is applied to the processor device 16 and the like. Further, in a case where the light emission timing of the illumination light is changed on a frame basis, there is a possibility that image processing may become complicated. Accordingly, since the first number of light emission frames or the second number of light emission frames is not changed to be fixed and only the first number of display frames or the second number of display frames is changed in this embodiment, a load is not applied to the processor device 16 and the like.
In a case where the first number of display frames or the second number of display frames is changed, the display control unit 66 performs control to display each observation image by each of the changed numbers of display frames. Here, in a case where the first number of display frames is set to “8” and the second number of display frames is set to “2” as a first display pattern where the first number of display frames is set to be larger than the second number of display frames as shown in
In
Here, the first observation images displayed in the light emission period of the second illumination light (the first observation images within a second illumination light-emission period) are displayed on the basis of the first observation images within a first illumination light-emission period that are obtained in the light emission period of the first illumination light prior to the light emission period of the second illumination light. For example, the first observation image within the second illumination light-emission period may be the first observation image obtained at the last fifth frame among the first observation images within the first illumination light-emission period of five frames. Further, the first observation image within the second illumination light-emission period may be an image that is obtained from the combination of the first observation images within the first illumination light-emission period of five frames. Furthermore, the first observation images within the second illumination light-emission period may be the same image or may be different images for all the three frames.
On the other hand, in a case where the first number of display frames is set to “2” and the second number of display frames is set to “8” as a second display pattern where the first number of display frames is set to be smaller than the second number of display frames as shown in
Here, it is preferable that the second observation images displayed in the light emission period of the first illumination light (the second observation images within the first illumination light-emission period) are generated on the basis of the second observation images within the second illumination light-emission period obtained in the light emission period of the second illumination light prior to the light emission period of the first illumination light and are displayed prior to the first observation images in the first illumination light-emission period. For example, it is preferable that the second observation image within the first illumination light-emission period may be generated on the basis of the second observation image obtained at the last fifth frame among the second observation images within the second illumination light-emission period. Further, the second observation image within the first illumination light-emission period may be an image that is obtained from the combination of all or some of images obtained in the second observation images within the second illumination light-emission period. Furthermore, the second observation images within the first illumination light-emission period may be the same image or may be different images for all the three frames.
The above-mentioned first and second display patterns (see
Further, in the above description, in a case where one of the first number of display frames and the second number of display frames is reduced, the other thereof has been increased by a reduction in one of the first number of display frames and the second number of display frames. However, the invention is not limited thereto. For example, in a case where the first number of display frames is reduced to “2” from “5” as shown in
Furthermore, in the above description, as shown in
Moreover, in the above description, the second observation image, which is obtained immediately after illumination light is switched to the second illumination light from the first illumination light, (the second observation image immediately after switching) is displayed as it is. However, since the second observation image immediately after switching is based on light where the first illumination light and the second illumination light are mixed, the second observation image immediately after switching is an image (an image where superficial blood vessels and medium-deep blood vessels are mixed) different from the original second observation image. For this reason, in a case where a user desires to clearly display a difference in specific subject information included in the first and second observation images, it is preferable that the second observation image immediately after switching (a second observation image obtained at the first frame in the light emission period of the second illumination light) is cancelled not to be displayed on the monitor 18 as shown in
Next, the multi-observation mode will be described along a flowchart of
After the user completes the change of the first number of display frames or the second number of display frames, the user operates the user interface unit 19 to end the setting of the number of display frames. Accordingly, the number-of-display-frame-setting menu 70 disappears from the monitor 18. Then, the multi-observation mode is performed. In accordance with this, the first illumination light is emitted for a time corresponding to the first number of light emission frames in a predetermined light emission cycle. After that, the first illumination light is switched to the second illumination light, and the second illumination light is emitted for a time corresponding to the second number of light emission frames. Further, in the light emission cycle, the first observation image based on the light emission of the first illumination light is displayed on the monitor 18 for a time corresponding to the first number of display frames. After that, the first observation image is switched to the second observation image, and the second observation image is displayed on the monitor 18 for a time corresponding to the second number of display frames. The light emission of the illumination light and the display of an object to be observed having been described above are repeatedly performed until the multi-observation mode ends.
In the embodiment, the first illumination light and the second illumination light have been emitted while being switched according to the specific light emission pattern, and the first observation image corresponding to the first illumination light and the second observation image corresponding to the second illumination light have been displayed on the monitor 18 while being switched according to the specific display pattern. However, three or more kinds of illumination light having wavelength ranges different from each other may be emitted while being switched according to a specific light emission pattern, and three or more kinds of observation images corresponding to the three or more kinds of illumination light may be displayed on the monitor 18 while being switched according to a specific display pattern.
Further, in the embodiment, as the specific light emission pattern, the first illumination light has been emitted for a time corresponding to the first number of light emission frames, the first illumination light then has been switched to the second illumination light, and the second illumination light has been emitted for a time corresponding to the second number of light emission frames in a predetermined light emission cycle. However, light may be emitted according to other light emission patterns. For example, in a case where third illumination light having a wavelength range between the wavelength of the first illumination light and the wavelength of the second illumination light is emitted in addition to the first illumination light and the second illumination light, as a specific light emission pattern, for example, the first illumination light, the third illumination light, and the second illumination light are emitted while being switched in this order and the number of light emission frames of any one of the first illumination light, the second illumination light, or the third illumination light is set to be larger than those of the other two.
Furthermore, in the embodiment, as the specific display pattern, the first observation image has been displayed on the monitor 18 for a time corresponding to the first number of display frames, the first observation image then has been switched to the second observation image, and the second observation image has been displayed on the monitor 18 for a time corresponding to the second number of display frames in a specific light emission cycle. However, images may be displayed according to other display patterns. For example, in a case where a third observation image obtained on the basis of third illumination light having a wavelength range between the wavelength of the first illumination light and the wavelength of the second illumination light is displayed in addition to the first observation image and the second observation image, as a specific display pattern, for example, the first observation image, the third observation image, and the second observation image are displayed while being switched in this order and the number of display frames of any one of the first observation image, the second observation image, or the third observation image is set to be larger than those of the other two.
The first number of display frames or the second number of display frames has been changed by a user in the embodiment, but the first number of display frames or the second number of display frames may be automatically changed on the basis of an observation image. For example, specific illumination light (of which the light intensities of violet light V and blue light B are higher than those of the other green light G and red light R), which can emphasize both superficial blood vessels and medium-deep blood vessels, is applied in addition to the first illumination light and the second illumination light, and a specific observation image, which is obtained in a case where an object to be observed is illuminated with the specific illumination light, is acquired. Then, a first difference image between the specific observation image and the first observation image and a second difference image between the specific observation image and the second observation image are obtained, and the first difference image and the second difference image are compared with each other. The first number of display frames and the second number of display frames may be determined on the basis of the result of this comparison.
For example, in a case where the first difference image has a pixel value (differential value) larger than the pixel value of the second difference image, many superficial blood vessels are included in the object to be observed. Accordingly, the first number of display frames or the second number of display frames is changed so that superficial blood vessels are emphasized. In contrast, in a case where the second difference image has a pixel value (differential value) larger than the pixel value of the first difference image, many medium-deep blood vessels are included in the object to be observed. Accordingly, the first number of display frames or the second number of display frames is changed so that medium-deep blood vessels are emphasized.
The hardware structures of the processing units, which are included in the processor device 16 in the embodiments, such as the image acquisition unit 53, the DSP 56, the noise removing unit 58, the image processing unit 60, the parameter switching unit 62, and the central control unit 68, are various processors to be described below. The various processors include: a central processing unit (CPU) that is a general-purpose processor functioning as various processing units by executing software (program); a programmable logic device (PLD) that is a processor of which circuit configuration can be changed after manufacture, such as a field programmable gate array (FPGA); a dedicated electrical circuit that is a processor having circuit configuration designed exclusively to perform various kinds of processing; and the like.
One processing unit may be formed of one of these various processors, or may be formed of a combination of two or more same kind or different kinds of processors (for example, a plurality of FPGAs or a combination of a CPU and an FPGA). Further, a plurality of processing units may be formed of one processor. As an example where a plurality of processing units are formed of one processor, first, there is an aspect where one processor is formed of a combination of one or more CPUs and software as typified by a computer, such as a client or a server, and functions as a plurality of processing units. Second, there is an aspect where a processor fulfilling the functions of the entire system, which includes a plurality of processing units, by one integrated circuit (IC) chip as typified by System On Chip (SoC) or the like is used. In this way, various processing units are formed using one or more of the above-mentioned various processors as hardware structures.
In addition, the hardware structures of these various processors are more specifically electrical circuitry where circuit elements, such as semiconductor elements, are combined.
[Additional claim 1]
An endoscope system comprising:
a plurality of semiconductor light sources that emit light having wavelength ranges different from each other;
a light source control unit that controls the semiconductor light sources, and performs control to emit a plurality of kinds of illumination light, which include first illumination light having a first light emission ratio and second illumination light having a second light emission ratio different from the first light emission ratio, while switching the plurality of kinds of illumination light according to a specific light emission pattern;
an image acquisition unit that acquires a plurality of observation images obtained from image pickup of an object to be observed illuminated with each illumination light, in which the plurality of observation images include a first observation image corresponding to the first illumination light and a second observation image corresponding to the second illumination light; and
a display control unit that performs control to display the plurality of observation images on a display unit while switching the plurality of observation images according to a specific display pattern,
wherein the specific light emission pattern is changeable and the specific display pattern is changeable.
[Additional claim 2]
The endoscope system according to Additional claim 1,
wherein the specific light emission pattern is a pattern where the first illumination light is emitted for a time corresponding to the first number of light emission frames, the first illumination light is then switched to the second illumination light, and the second illumination light is emitted for a time corresponding to the second number of light emission frames in a predetermined specific light emission cycle,
the specific display pattern is a pattern where the first observation image is displayed on the display unit for a time corresponding to the first number of display frames, the first observation image is then switched to the second observation image, and the second observation image is displayed on the display unit for a time corresponding to the second number of display frames in the specific light emission cycle, and
the first number of light emission frames and the second number of light emission frames are changeable, and the first number of display frames and the second number of display frames are changeable.
| Number | Date | Country | Kind |
|---|---|---|---|
| JP2017-217889 | Nov 2017 | JP | national |
This application is a Continuation of PCT International Application No. PCT/JP2018/041288 filed on 7 Nov. 2018, which claims priority under 35 U.S.C. § 119(a) to Japanese Patent Application No. 2017-217889 filed on 13 Nov. 2017. The above application is hereby expressly incorporated by reference, in its entirety, into the present application.
| Number | Name | Date | Kind |
|---|---|---|---|
| 20060209185 | Yokoi | Sep 2006 | A1 |
| 20120157775 | Yamaguchi | Jun 2012 | A1 |
| 20120253158 | Yamaguchi | Oct 2012 | A1 |
| 20130245411 | Saito | Sep 2013 | A1 |
| 20150087903 | Kuramoto | Mar 2015 | A1 |
| 20150245002 | Kuramoto | Aug 2015 | A1 |
| 20160007829 | Chun | Jan 2016 | A1 |
| 20160296106 | Shoji | Oct 2016 | A1 |
| 20170112370 | Daidoji et al. | Apr 2017 | A1 |
| 20170231480 | Yamazaki | Aug 2017 | A1 |
| 20180214009 | Endo | Aug 2018 | A1 |
| 20190117055 | Ito | Apr 2019 | A1 |
| 20190222737 | Aoyama | Jul 2019 | A1 |
| Number | Date | Country |
|---|---|---|
| 1882275 | Dec 2006 | CN |
| 102613953 | Aug 2012 | CN |
| 102727217 | Oct 2012 | CN |
| 102860809 | Jan 2013 | CN |
| 104434000 | Mar 2015 | CN |
| 105992546 | Oct 2016 | CN |
| 106535739 | Mar 2017 | CN |
| 107072508 | Aug 2017 | CN |
| 2 505 141 | Oct 2012 | EP |
| 3 520 673 | Aug 2019 | EP |
| 2001-149303 | Jun 2001 | JP |
| 2005-143991 | Jun 2005 | JP |
| 2012-110485 | Jun 2012 | JP |
| 2012110485 | Jun 2012 | JP |
| 2012-213551 | Nov 2012 | JP |
| 2013-188364 | Sep 2013 | JP |
| 2015-159957 | Sep 2015 | JP |
| 2016-055052 | Apr 2016 | JP |
| 2016055052 | Apr 2016 | JP |
| 2017-185258 | Oct 2017 | JP |
| 2014156938 | Oct 2014 | WO |
| 2017057392 | Apr 2017 | WO |
| 2018066347 | Apr 2018 | WO |
| Entry |
|---|
| An Office Action; “Notice of Reasons for Refusal”, mailed by the Japanese Patent Office dated May 11, 2021, which corresponds to Japanese Patent Application No. 2019-552830 and is related to U.S. Appl. No. 16/869,895; with English language translation. |
| International Search Report issued in PCT/JP2018/041288; dated Jan. 29, 2019. |
| International Preliminary Report on Patentability issued in PCT/JP2018/041288; completed Dec. 12, 2019. |
| The extended European search report issued by the European Patent Office dated Dec. 4, 2020, which corresponds to European Patent Application No. 18876807.1-1122 and is related to U.S. Appl. No. 16/869,895. |
| An Office Action; “Notice of Reasons for Refusal”, mailed by the Japanese Patent Office dated Oct. 26, 2021, which corresponds to Japanese Patent Application No. 2019-552830 and is related to U.S. Appl. No. 16/869,895; with English language translation. |
| An Office Action; “Trial and Appeal Decision”, mailed by the Japanese Patent Office dated Jul. 5, 2022, which corresponds to Japanese Patent Application No. 2019-552830 and is related to U.S. Appl. No. 16/869,895; with English language translation. |
| An Office Action mailed by China National Intellectual Property Administration dated Oct. 9, 2022, which corresponds to Chinese Patent Application No. 201880073237.9 and is related to U.S. Appl. No. 16/869,895; with English language translation. |
| Number | Date | Country | |
|---|---|---|---|
| 20200260942 A1 | Aug 2020 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/JP2018/041288 | Nov 2018 | US |
| Child | 16869895 | US |
