This application claims priority under 35 USC 119 from Japanese Patent Application No. 2016-197542, filed 5 Oct. 2016, the disclosure of which is incorporated by reference herein.
The present invention relates to an endoscope system that can perform direct-viewing observation and side-viewing observation and a method of driving the endoscope system.
Diagnosis, which is performed using an endoscope system including a light source device, an endoscope, and a processor device, has become common in the field of medicine. In the endoscope system, the light source device generates illumination light. The endoscope includes a flexible insertion part, and the insertion part is inserted into a subject to pick up an image of an object to be observed by, for example, an image sensor that is mounted on a tip portion (hereinafter, referred to as a tip part) of the insertion part. Further, the processor device generates the image of the object to be observed, and displays the image on a monitor.
A direct-viewing observation type endoscope that picks up an image of an object to be observed present in a tip direction of a tip part (that is, the direction of the front along an insertion direction of an insertion part) and a side-viewing observation type endoscope that picks up an image of an object to be observed present in a lateral direction of a tip part (that is, the circumferential direction of an insertion part) are known as an endoscope used in an endoscope system in the related art. Further, endoscopes that can observe objects in both the tip direction and the lateral direction of a tip part have been known in recent years (WO2011/055641A (corresponding to US2011/282148A), JP2016-007745A, and JP2012-245157A (corresponding to US2014/046131A)).
A direct-viewing observation image that is obtained from the image pick-up of an object to be observed present in the tip direction of the tip part and a side-viewing observation image that is obtained from the image pick-up of an object to be observed present in the lateral direction of the tip part are obtained in the endoscope that can perform direct-viewing observation and side-viewing observation. For this reason, an observable range in a case in which the direct-viewing observation image and the side-viewing observation image are displayed is wider than an observable range in a case in which only one of the direct-viewing observation image and the side-viewing observation image is displayed. However, it may not be said that the endoscope is necessarily easy to use in observation, diagnosis, or the like in a case in which the direct-viewing observation image and the side-viewing observation image are merely displayed.
An object of the invention is to provide an endoscope system that is easy to use in observation, diagnosis, or the like since a method of displaying a direct-viewing observation image and a side-viewing observation image is improved in a case in which an endoscope capable of performing direct-viewing observation and side-viewing observation is used, and a method of driving the endoscope system.
An endoscope system of the invention comprises: an endoscope including a side-viewing observation unit that includes a field of view in a lateral direction of an insertion part to be inserted into an object to be observed and a protruding portion that protrudes from the insertion part and forms a blind spot in the field of view of the side-viewing observation unit; an image acquisition unit that acquires a side-viewing observation image by using the side-viewing observation unit; a display unit that displays the side-viewing observation image; and a display control section that allows at least a part of the blind spot of the side-viewing observation image in which the protruding portion is shown up not to be displayed, and displays the side-viewing observation image on the display unit.
It is preferable that the display control section allows at least a part of the blind spot not to be displayed by offsetting a display position of the side-viewing observation image with respect to a display region on the display unit.
It is preferable that the display control section allows at least a part of the blind spot not to be displayed by enlarging the side-viewing observation image with respect to the display region on the display unit.
It is preferable that the display control section includes a blind spot-non-display mode as a display mode for the side-viewing observation image and allows at least a part of the blind spot not to be displayed and displays the side-viewing observation image on the display unit in a case in which the display mode is set to the blind spot-non-display mode.
It is preferable that the display control section allows the entire blind spot not to be displayed.
It is preferable that the display control section changes a ratio of a portion, which is not to be displayed, of the blind spot.
It is preferable that the display control section changes the ratio of the portion, which is not to be displayed, of the blind spot in accordance with the movement of the insertion part.
It is preferable that the display control section changes the ratio of the portion, which is not to be displayed, of the blind spot in a case in which the insertion part is to be inserted into the object to be observed and a case in which the insertion part is to be extracted from the object to be observed.
It is preferable that the endoscope includes a direct-viewing observation unit including a field of view in a tip direction of the insertion part, the image acquisition unit acquires a direct-viewing observation image by using the direct-viewing observation unit, and the display control section adjusts display ratios of the side-viewing observation image and the direct-viewing observation image and changes the ratio of the portion, which is not to be displayed, of the blind spot in accordance with the display ratios of the side-viewing observation image and the direct-viewing observation image.
It is preferable that the endoscope system further includes a nozzle that ejects a washing substance to the side-viewing observation unit to wash the side-viewing observation unit and is provided in a non-display region not to be displayed in a case in which the display control section displays the side-viewing observation image on the display unit.
The invention provides a method of driving an endoscope system. the endoscope system includes an endoscope including a side-viewing observation unit that includes a field of view in a lateral direction of an insertion part to be inserted into an object to be observed and a protruding portion that protrudes from the insertion part and forms a blind spot in the field of view of the side-viewing observation unit, an image acquisition unit that acquires a side-viewing observation image by using the side-viewing observation unit, and a display unit that displays the side-viewing observation image. The method comprises: a step of allowing at least a part of the blind spot of the side-viewing observation image in which the protruding portion is shown up not to be displayed, and displaying the side-viewing observation image on the display unit by the display control section.
The invention can provide an endoscope system that is easy to use in observation, diagnosis, or the like since a method of displaying a direct-viewing observation image and a side-viewing observation image is improved in a case in which an endoscope capable of performing direct-viewing observation and side-viewing observation is used, and a method of driving the endoscope system.
As shown in
The endoscope 12 includes an insertion part 12a that is to be inserted into a subject, an operation unit 12b that is provided at a base end portion of the insertion part 12a, a bendable part 12c that is provided on the tip side of the insertion part 12a, and a tip part 12d. In a case in which an angle knob 12e provided on the operation unit 12b is operated, the bendable part 12c is bent. As a result of the bend of the bendable part 12c, the tip part 12d faces a desired direction.
Further, the operation unit 12b includes, for example, a washing switch 13, which is used to eject washing liquid from nozzles provided in the tip part 12d, in addition to the angle knob 12e. In a case in which the washing switch 13 is pressed in a state in which dirt adheres to the tip part 12d due to contact between the object to be observed and the tip part 12d or the like, the washing liquid is ejected toward at least a part of the tip part 12d from the nozzles provided in the tip part 12d. As a result, a portion of the tip part 12d to which the washing liquid is ejected can be washed. In the endoscope system 10, the washing liquid is liquid, such as water or liquid medicine. Further, for convenience sake, washing “liquid” is described in this specification. However, gas, such as air, solid, a mixture of materials having different phases, and the like, which are to be ejected from the nozzles, are also included in the “washing liquid”, as long as they are used for washing.
As shown in
The first protruding portion 31 has a substantially cylindrical shape as a whole, and includes a direct-viewing observation window 41A that is provided at the tip thereof and is an observation window for a direct-viewing observation unit 41 and a side-viewing observation window 42A that is provided on the side thereof and is an observation window for a side-viewing observation unit 42. The direct-viewing observation unit 41 has the field of view in the tip direction of the insertion part 12a, and picks up an image of an object to be observed that is present in the tip direction of the insertion part 12a. The direct-viewing observation unit 41 includes, for example, an image pickup lens, an image sensor, and the like. An optical member, such as the image pickup lens of the direct-viewing observation unit 41, or a transparent protective member that protects the optical member, such as the image pickup lens, is exposed to the tip (the surface facing the Z direction) of the first protruding portion 31. A portion, which is exposed to the tip of the first protruding portion 31, is the direct-viewing observation window 41A that takes in light incident from the object to be observed present in the tip direction of the insertion part 12a.
The side-viewing observation unit 42 has the field of view in a lateral direction of the insertion part 12a, and picks up an image of an object to be observed that is present in the lateral direction of the insertion part 12a. The side-viewing observation unit 42 includes, for example, an image pickup lens, an image sensor, and the like as in the case of the direct-viewing observation unit 41. An optical member, such as the image pickup lens of the side-viewing observation unit 42, or a transparent protective member that protects the optical member, such as the image pickup lens, is exposed to the side surface of the first protruding portion 31 (the surface forming the outer periphery of the first protruding portion 31). A portion, which is exposed to the side surface of the first protruding portion 31, is the side-viewing observation window 42A that takes in light incident from the object to be observed present in the lateral direction of the insertion part 12a. In the endoscope 12 of this embodiment, the side-viewing observation unit 42 is exposed over the circumference of the first protruding portion 31, which excludes a joint portion between the first protruding portion 31 and the second protruding portion 32, in the circumferential direction of the first protruding portion 31 and forms the belt-like side-viewing observation window 42A.
Further, the first protruding portion 31 includes a first side-viewing illumination window 43A, which is an illumination window for a side-viewing illumination unit 43, in addition to the direct-viewing observation window 41A and the side-viewing observation window 42A. The side-viewing illumination unit 43 emits illumination light toward the field of view of the side-viewing observation unit 42 from the first side-viewing illumination window 43A. The side-viewing illumination unit 43 includes, for example, a light guide that guides the illumination light emitted from the light source device 14, and an optical member, such as a lens or a mirror, that diffuses and emits the illumination light guided to the tip part 12d by the light guide toward the field of view of the side-viewing observation unit 42. The optical member, such as the mirror of the side-viewing illumination unit 43, or a transparent protective member that protects the optical member, such as the mirror, is exposed to the side surface of the first protruding portion 31. A portion, which is exposed to the side surface of the first protruding portion 31, is the first side-viewing illumination window 43A that emits illumination light in the lateral direction of the insertion part 12a. In the endoscope 12 of this embodiment, a part of the outer periphery of the first protruding portion 31, which excludes the joint portion between the first protruding portion 31 and the second protruding portion 32, forms side-viewing illumination windows. Further, the first side-viewing illumination window 43A is provided on the right side surface of the first protruding portion 31 in
In this embodiment, the direct-viewing observation unit 41 and the side-viewing observation unit 42 includes a common image pickup lens 61 and a common image sensor 66 as shown in
The mirror lens 63 guides light, which is incident through the front group lens 62 from an object to be observed present in the tip direction of the insertion part 12a, to the rear group lens 64. Then, the light forms an image on the image pickup surface of the image sensor 66 through a cover glass 67. Accordingly, the image pickup lens 61 and the image sensor 66 serving as the direct-viewing observation unit 41 pick up an image of the object to be observed that is present in the tip direction of the insertion part 12a.
On the other hand, the mirror lens 63 sequentially reflects light, which is incident from an object to be observed present in the lateral direction of the insertion part 12a through the side surface of the mirror lens 63, by a joint surface between the two lenses of the mirror lens 63 and the front surface of the mirror lens 63, and guides the light to the rear group lens 64. Then, the light forms an image on the image pickup surface of the image sensor 66 through the cover glass 67. Accordingly, the image pickup lens 61 and the image sensor 66 serving as the side-viewing observation unit 42 pick up an image of the object to be observed that is present in the lateral direction of the insertion part 12a.
Further, the side-viewing illumination unit 43 includes a light guide 71, a reflective member 72, and a filling member 73. The light guide 71 is optically connected to the light source device 14, and guides illumination light that is emitted from the light source device 14. Then, the illumination light is emitted to the reflective member 72 from the end face of the light guide 71 through the filling member 73. The reflective member 72 diffuses the illumination light, which is incident from the light guide 71, in the lateral direction of the insertion part 12a, and emits the illumination light to a range that includes the field of view of at least the side-viewing observation unit 42. The filling member 73 is a protective member that protects the emission end face of the light guide 71 and the reflective member 72, and is transparent. Furthermore, the filling member 73 smoothly fills a groove portion, which is formed between the light guide 71 and the reflective member 72, along the side surface of the first protruding portion 31. For this reason, the filling member 73 forms the first side-viewing illumination window 43A. The second side-viewing illumination window 43B is also the same as described above.
In the endoscope 12 of this embodiment, the side-viewing observation window 42A is provided on the side surface of the first protruding portion 31 so as to be close to the tip of the first protruding portion 31, and the first side-viewing illumination window 43A and the second side-viewing illumination window 43B are provided on the side surface of the first protruding portion 31 so as to be close to the base end of the first protruding portion 31. However, the positions and the order of the side-viewing observation window 42A, the first side-viewing illumination window 43A, and the second side-viewing illumination window 43B are arbitrary. Since vignetting or the like caused by the tip surface 21 or the like is prevented and the field of view of the side-viewing illumination unit 43 is easily ensured in a case in which the side-viewing observation window 42A is provided on the side surface of the first protruding portion 31 so as to be close to the tip of the first protruding portion 31, it is good that the side-viewing observation window 42A is provided on the side surface of the first protruding portion 31 so as to be close to the tip of the first protruding portion 31 as much as possible.
The second protruding portion 32 includes nozzles that eject washing liquid to wash the tip part 12d. More specifically, the second protruding portion 32 includes a nozzle 51 and a nozzle 52 (see
The second protruding portion 32 further includes a nozzle 53 at the tip of the second protruding portion 32. The nozzle 53 washes the direct-viewing observation window 41A by ejecting washing liquid toward the direct-viewing observation window 41A that is an exposed portion of the direct-viewing observation unit 41.
As shown in
The second protruding portion 32 includes a direct-viewing illumination window 54A, which is an illumination window for a direct-viewing illumination unit 54 for emitting illumination light toward the field of view of the direct-viewing observation unit 41, in addition to the nozzles 51, 52, and 53. The direct-viewing illumination unit 54 includes, for example, a light guide 77 that guides illumination light emitted from the light source device 14, and an illumination lens 78 and the like that diffuse and emit the illumination light, which is guided to the tip part 12d by the light guide 77, toward the field of view of the direct-viewing observation unit 41 (see
The tip surface 21 of the tip part 12d includes a direct-viewing illumination window 81A, which is an illumination window for a direct-viewing illumination unit 81, and a forceps port 82 in addition to the first and second protruding portions 31 and 32.
The direct-viewing illumination unit 81 emits illumination light toward the field of view of the direct-viewing observation unit 41 as in the case of the direct-viewing illumination unit 54 that is provided in the second protruding portion 32. Further, the direct-viewing illumination unit 81 includes, for example, a light guide 84 (see
The forceps port 82 is an outlet for a treatment tool, such as forceps. In a case in which a treatment tool, such as forceps, is inserted from an inlet port (not shown) that is provided at the base end portion of the endoscope 12, the treatment tool reaches the forceps port 82 through a forceps channel and the tip of the treatment tool can protrude from the forceps port 82. The forceps channel communicates with the tip part 12d, the insertion part 12a, and the operation unit 12b.
Since the side-viewing observation window 42A is provided on the side surface of the first protruding portion 31 and the second protruding portion 32 is adjacent to the first protruding portion 31 and protrudes from the tip surface 21 in the tip direction of the insertion part 12a (the Z direction), the second protruding portion 32 is a protruding portion that forms the blind spot of the field of view of the side-viewing observation unit 42. That is, since the second protruding portion 32 is within a part of the field of view of the side-viewing observation unit 42 due to the disposition of the respective units, an image of an object to be observed cannot be picked up in a part of an angular range.
As shown in
The light source control unit 92 controls each of the turn-on, the turn-off, and the amount of light of the LEDs or the like of the light source 91 in accordance with a drive timing of the image sensor 66. Particularly, in a case in which one observation image is generated using a plurality of picked-up images (that is, a multi-frame observation mode), the light source control unit 92 can change the wavelength band or a spectrum of the illumination light for every image pickup frame where a plurality of picked-up images to be used for the generation of an observation image is obtained, as a result of the control of the LEDs. The turn-on means that the amount of light allowing the image sensor 66 to pick up an image of an object to be observed (the amount of light for allowing an image of an object to be observed to be visually recognized in an observation image) is emitted. The turn-on includes a case in which light is reduced to the amount of light not allowing the image sensor 66 to pick up an image of an object to be observed in addition to a case in which light emission is completely stopped. Further, in a case in which each of the light sources of the light source 91 is a semiconductor light source, the light source control unit 92 controls the turn-on, the turn-off, and the amount of light of each light source by pulse modulation control.
Illumination light, which is emitted from the light source 91, is incident on a light guide 93. Since the light guide 93 is inserted into the endoscope 12 and the universal cord from the light source device 14, the light guide 93 transmits illumination light up to the tip part 12d of the endoscope 12. The light guide 93 is branched to at least the light guide 77 of the direct-viewing illumination unit 54, the light guide 84 of the direct-viewing illumination unit 81, and the light guide 71 of the side-viewing illumination unit 43, and transmits illumination light to each of these illumination units. A multimode fiber can be used as the light guide 93 and each branched light guide, such as the light guide 71. For example, a thin fiber cable of which a total diameter of a core diameter of 105 μm, a cladding diameter of 125 μm, and a protective layer forming a covering is in the range of φ 0.3 to 0.5 mm can be used.
The processor device 16 includes a control unit 96, an image acquisition unit 97, and a display control section 98. The control unit 96 is a central processing unit (CPU) that generally controls the endoscope system 10. Each of other processing units (such as the image acquisition unit and the display control section) is configured by the CPU executing corresponding program or by a dedicated processing circuitry. The control unit 96 performs, for example, synchronization control for allowing an image pickup timing of the image sensor 66 to correspond to an image pickup timing of each LED of the light source 91. The control of the image pickup timing of each LED of the light source 91 is performed through the light source control unit 92. Further, the control unit 96 performs automatic exposure control (AE control) for automatically controlling exposure. In this embodiment, the control unit 96 performs AE control by driving the image sensor 66 at a fixed timing and adjusting the amount of light (that is, the amount of illumination light) emitted from each LED or the like of the light source 91. During AE control, the control unit 96 acquires a direct-viewing observation image 111 (see
The image acquisition unit 97 acquires the direct-viewing observation image 111 by the direct-viewing observation unit 41, and acquires the side-viewing observation image 112 by the side-viewing observation unit 42. In this embodiment, the direct-viewing observation unit 41 and the side-viewing observation unit 42 share the image pickup lens 61 and the image sensor 66. Accordingly, the image acquisition unit 97 acquires an image that is picked up by the image sensor 66 (hereinafter, referred to as a picked-up image), and obtains a picked-up image, which includes the direct-viewing observation image 111 and the side-viewing observation image 112, by various kinds of image processing and the like on the acquired picked-up image. That is, a part of the picked-up image, which is obtained from the image sensor 66, is the direct-viewing observation image 111, and the other part of the picked-up image, which is obtained from the image sensor 66, is the side-viewing observation image 112.
The image acquisition unit 97 functions as, for example, a digital signal processor (DSP) and a noise reduction unit.
The image acquisition unit 97 performs various kinds of processing, such as defect correction processing, offset processing, gain correction processing, linear matrix processing, gamma conversion processing, demosaic processing, and YC conversion processing, on the acquired picked-up image as necessary. This is the function of the image acquisition unit 97 as the DSP. The defect correction processing is processing for correcting the pixel value of a pixel corresponding to a defective pixel of the image sensor 66. The offset processing is processing for reducing a dark current component from the picked-up image having been subjected to the defect correction processing and setting an accurate zero level. The gain correction processing is processing for adjusting the signal level of each picked-up image by multiplying the picked-up image, which has been subjected to the offset processing, by a gain. The linear matrix processing is processing for improving the color reproducibility of the picked-up image subjected to the offset processing, and the gamma conversion processing is processing for adjusting the brightness or chroma of the picked-up image having been subjected to the linear matrix processing. The demosaic processing (also referred to as equalization processing or synchronization processing) is processing for interpolating the pixel value of a missing pixel, and is performed on the picked-up image having been subjected to the gamma conversion processing. The missing pixel is a pixel that does not have a pixel value since pixels having other colors are disposed in the image sensor 66 due to the arrangement of color filters. For example, since a B image is a picked-up image that is obtained from the image pick-up of the object to be observed at a B pixel, pixels of the B image, which are present at positions corresponding to a G pixel and an R pixel of the image sensor 66, does not have a pixel value. In the demosaic processing, the pixel values of pixels of a B image, which are present at positions corresponding to a G pixel and an R pixel of the image sensor 66, are generated through the interpolation of the B image. The YC conversion processing is processing for converting an image, which has been subjected to the demosaic processing, into a luminance channel Y, a color-difference channel Cb, and a color-difference channel Cr.
The image acquisition unit 97 performs noise-reduction processing on the luminance channel Y, the color-difference channel Cb, and the color-difference channel Cr by using, for example, a moving-average method, a median filter method, or the like. A conversion unit 59 converts the luminance channel Y, the color-difference channel Cb, and the color-difference channel Cr, which have been subjected to the noise-reduction processing, into a picked-up image having the respective colors of BGR again. This is the function of the image acquisition unit 97 as the noise reduction unit.
The display control section 98 acquires the direct-viewing observation image 111 and the side-viewing observation image 112 from the image acquisition unit 97. Then, the display control section 98 generates an image for display (hereinafter, referred to as a display image) (see
Further, the display control section 98 masks a portion of the side-viewing observation image 112 corresponding to a blind spot, and adjusts a relative display ratio of the direct-viewing observation image 111 and the side-viewing observation image 112 on the monitor 18 serving as the display unit. For this reason, as shown in
The display ratio setting unit 101 includes a plurality of display modes as the setting of the relative display ratio in advance. Specifically, the display ratio setting unit 101 includes three kinds of display modes, that is, a standard mode, a direct-viewing enlargement mode, and a side-viewing enlargement mode.
The standard mode is a display mode in which both the direct-viewing observation image 111 and the side-viewing observation image 112 are substantially not enlarged or reduced and a relative display ratio of the direct-viewing observation image 111 and the side-viewing observation image 112 is set to a display ratio originally obtained when an image is picked up. That is, in the standard mode, a relative display ratio of the direct-viewing observation image 111 and the side-viewing observation image 112 becomes substantially a display ratio that is obtained when an image is picked up. “Substantially not enlarged or reduced” means the exclusion of the enlargement or reduction of the direct-viewing observation image 111 or the side-viewing observation image 112 that is inevitable for the connection between the direct-viewing observation image 111 and the side-viewing observation image 112. Accordingly, even in the standard mode, there is a case where any one or both of the direct-viewing observation image 111 and the side-viewing observation image 112 is enlarged or reduced for connection. Hereinafter, for simplification, it is regarded in the standard mode that the display area of the direct-viewing observation image 111 on the monitor 18 serving as the display unit and the display area of the side-viewing observation image 112 on the monitor 18 are substantially equal to each other.
The direct-viewing enlargement mode is a display mode in which the display area of the direct-viewing observation image 111 on the monitor 18 serving as the display unit is enlarged in comparison with the case of the standard mode. That is, in the case of the direct-viewing enlargement mode, the display ratio setting unit 101 relatively increases the display ratio of the direct-viewing observation image 111 to the side-viewing observation image 112. Further, the direct-viewing enlargement mode includes a case in which only the direct-viewing observation image 111 of the direct-viewing observation image 111 and the side-viewing observation image 112 is substantially displayed.
The side-viewing enlargement mode is a display mode in which the display area of the side-viewing observation image 112 on the monitor 18 serving as the display unit is enlarged in comparison with the case of the standard mode. That is, in the case of the side-viewing enlargement mode, the display ratio setting unit 101 relatively increases the display ratio of the side-viewing observation image 112 to the direct-viewing observation image 111. Further, the side-viewing enlargement mode includes a case in which only the side-viewing observation image 112 of the direct-viewing observation image 111 and the side-viewing observation image 112 is substantially displayed.
A doctor or the like who is a user of the endoscope system 10 can arbitrarily set a specific display ratio of the direct-viewing observation image 111 in the direct-viewing enlargement mode and the side-viewing observation image 112 in the side-viewing enlargement mode by using, for example, the console 19, the switch of the operation unit 12b, or an input device (hereinafter, referred to as the console 19), such as a foot switch. The specific display ratio of the direct-viewing observation image 111 and the side-viewing observation image 112 can be arbitrarily changed even during observation. The side-viewing enlargement mode is automatically made in a case in which a doctor or the like sets a numerical value or the like for allowing the display area of the direct-viewing observation image 111 in the direct-viewing enlargement mode to be smaller than that in the standard mode; and vice versa. Further, the standard mode, the direct-viewing enlargement mode, and the side-viewing enlargement mode can also be arbitrarily switched by the console 19 or the like, likewise.
The display image generating unit 102 generates a display image by enlarging or reducing the direct-viewing observation image 111 and the side-viewing observation image 112 or any one of the direct-viewing observation image 111 and the side-viewing observation image 112 as necessary in accordance with the display ratio that is set by the display ratio setting unit 101, connecting the direct-viewing observation image 111 to the side-viewing observation image 112, and masking a portion of the side-viewing observation image 112 corresponding to a blind spot.
“Mask” means that data of the side-viewing observation image 112 is modulated or replaced with other images or the like. A portion of the side-viewing observation image 112 corresponding to a blind spot is a portion at which an image of an object to be observed cannot be picked up due to the presence of the second protruding portion 32. Hereinafter, in the display image, a portion of the side-viewing observation image 112, which corresponds to a blind spot caused by the presence of the second protruding portion 32, is referred to as a first blind spot portion. The display image generating unit 102 masks at least the first blind spot portion in the display image. In addition, in this embodiment, the display image generating unit 102 also masks a portion (hereinafter, referred to as a second blind spot portion) of the side-viewing observation image 112, which is positioned outside the field of view of the side-viewing observation unit 42 and at which an image of an object to be observed cannot be picked up from the beginning, in the display image. The display control section 98 displays a display image, which is generated by the display image generating unit 102, on the monitor 18.
In the endoscope system 10, a display image of which a relative display ratio of the direct-viewing observation image 111 and the side-viewing observation image 112 is adjusted is displayed on the monitor 18. For example, in a case in which a display mode is set to the standard mode, the display control section 98 generates a display image 113 shown in
Actually, there is no clear boundary between the direct-viewing observation image 111 and the side-viewing observation image 112, but a boundary is shown in
In a case in which a display mode is set to the direct-viewing enlargement mode, the display control section 98 generates a display image 123 shown in
Accordingly, a display range (the range of an object to be observed, which can be displayed on the monitor 18, with respect to the entire object to be observed) and a display area (the area of a portion of the monitor 18 displaying the object to be observed) of the object to be observed in the display image 123 in the direct-viewing enlargement mode are the same as those in the display image 113 in the standard mode, but the display image 123 in the direct-viewing enlargement mode has a relatively large display ratio of the direct-viewing observation image 111 to the side-viewing observation image 112 in the display range and the display area of the object to be observed. For this reason, in a case in which a display mode is set to the direct-viewing enlargement mode, a doctor or the like can observe an object to be observed in a wide range by the direct-viewing observation image 111 and the side-viewing observation image 112 and particularly easily observes the object to be observed shown up in the direct-viewing observation image 111. For example, if a display mode is set to the direct-viewing enlargement mode in a case in which a portion to be noted, such as a portion suspected as a lesion, is discovered in the tip direction of the insertion part 12a, a doctor or the like can observe the portion to be noted in more detail.
Further, in a case in which a display mode is set to the side-viewing enlargement mode, the display control section 98 generates a display image 126 shown in
Accordingly, a display range and a display area of the object to be observed in the display image 126 in the side-viewing enlargement mode are the same as those in the display image 113 in the standard mode, but the display image 126 in the side-viewing enlargement mode has a large display ratio of the side-viewing observation image 112 to the direct-viewing observation image 111 in the display range and the display area of the object to be observed. For this reason, in a case in which a display mode is set to the side-viewing enlargement mode, a doctor or the like can observe an object to be observed in a wide range by the direct-viewing observation image 111 and the side-viewing observation image 112 and particularly easily observes the object to be observed shown up in the side-viewing observation image 112. For example, if a display mode is set to the side-viewing enlargement mode in a case in which a portion to be noted, such as a portion suspected as a lesion, is discovered in the lateral direction of the insertion part 12a, a doctor or the like can observe the portion to be noted in more detail.
As described above, the endoscope system 10 has the direct-viewing enlargement mode and the side-viewing enlargement mode in which a relative display ratio of the direct-viewing observation image 111 and the side-viewing observation image 112 on the monitor 18 is adjusted. Further, in a case in which a display mode is switched to the direct-viewing enlargement mode or the side-viewing enlargement mode, an object to be observed, which is shown up in the direct-viewing observation image 111 or the side-viewing observation image 112, can be enlarged while the display range and the display area of an object to be observed are maintained. For this reason, the endoscope system 10 is more easily used for observation, diagnosis, or the like than in the related art.
In the direct-viewing enlargement mode, for example, the display ratio of the direct-viewing observation image 111 can be set to 100% and the display ratio of the side-viewing observation image 112 can be set to 0%. In this case, a display image 128 in which substantially only the direct-viewing observation image 111 is displayed as shown in
Further, in the side-viewing enlargement mode, for example, the display ratio of the side-viewing observation image 112 can be set to 100% and the display ratio of the direct-viewing observation image 111 can be set to 0%. In this case, a display image 129 in which substantially only the side-viewing observation image 112 is displayed as shown in
In the first embodiment, a doctor or the like who is a user of the endoscope system 10 sets a relative display ratio of the direct-viewing observation image 111 in the direct-viewing enlargement mode and the side-viewing observation image 112 in the side-viewing enlargement mode. However, a relative display ratio of the direct-viewing observation image 111 in the direct-viewing enlargement mode and the side-viewing observation image 112 in the side-viewing enlargement mode can be automatically set. In this case, the processor device 16 is provided with a movement detecting unit 201 as in an endoscope system 200 shown in
The movement detecting unit 201 detects the movement of the tip part 12d of the insertion part 12a. The movement of the tip part 12d is the moving direction of the tip part 12d along an object to be observed, a change in the position (velocity, speed, or acceleration) of the tip part 12d in the moving direction of the tip part 12d along an object to be observed, the orientation of the tip part 12d caused by a bend of the bendable part 12c, the deflection angle of the tip part 12d caused by a bend of the bendable part 12c, a change in the orientation (angular speed or angular acceleration) of the tip part 12d caused by a bend of the bendable part 12c, or the like. The movement sensor 202 is a sensor for detecting the movement of the tip part 12d, and is, for example, a speed sensor, an acceleration sensor, an angular speed sensor, an angular acceleration sensor, a sensor or a mechanism for measuring the length of a portion of the insertion part 12a inserted into an object to be observed, a sensor or a mechanism for measuring the degree of a bend of the bendable part 12c, or the like.
For example, the movement detecting unit 201 acquires a plurality of direct-viewing observation images 111 and a plurality of side-viewing observation images 112 or any one of the plurality of direct-viewing observation images 111 and the plurality of side-viewing observation images 112 from an image acquisition unit 97 with time, and detects the movement of the tip part 12d by using the acquired direct-viewing observation images 111 or the acquired side-viewing observation images 112. Further, in a case in which the endoscope 12 is provided with the movement sensor 202, the movement of an object to be observed is detected using an output signal of the movement sensor 202 instead of the direct-viewing observation images 111 or the side-viewing observation images 112 or in addition to the direct-viewing observation images 111 or the side-viewing observation images 112.
Further, in a case in which the processor device 16 includes the movement detecting unit 201, the display control section 98 adjusts a relative display ratio of the direct-viewing observation image 111 and the side-viewing observation image 112 with reference to the movement of the tip part 12d detected by the movement detecting unit 201. That is, the display control section 98 automatically selects a display mode due to the movement of the tip part 12d, which is detected by the movement detecting unit 201, in the display ratio setting unit 101. Furthermore, in a case in which the automatically selected display mode is the direct-viewing enlargement mode or the side-viewing enlargement mode, the display control section 98 also automatically determines a relative display ratio of the direct-viewing observation image 111 and the side-viewing observation image due to the movement of the tip part 12d that is detected by the movement detecting unit 201.
In the endoscope system 200, for example, the display control section 98 can adjust a relative display ratio of the direct-viewing observation image 111 and the side-viewing observation image 112 in accordance with the insertion/extraction of the insertion part 12a. In this case, as shown in
If the moving direction of the tip part 12d is the insertion direction (YES in S211), the display control section 98 automatically sets a display mode to the direct-viewing enlargement mode in the display ratio setting unit 101 (S212) and displays the display image 123 in the direct-viewing enlargement mode on the monitor 18. On the other hand, if the moving direction of the tip part 12d is the extraction direction (NO in S211), the display control section 98 automatically sets a display mode to the side-viewing enlargement mode in the display ratio setting unit 101 (S213) and displays the display image 126 in the side-viewing enlargement mode on the monitor 18.
In the endoscope system 200, as described above, a display mode is set to the direct-viewing enlargement mode in a case in which the insertion part 12a is to be inserted, and a display mode is set to the side-viewing enlargement mode in a case in which the insertion part 12a is to be extracted. Accordingly, the display ratio of an image, which is likely to be noted by a doctor or the like in a case in which the insertion part 12a is to be inserted or extracted, of the direct-viewing observation image 111 and the side-viewing observation image 112 on the monitor 18 is automatically increased. For this reason, the insertion part 12a of the endoscope system 200 is more easily inserted/extracted than an insertion part of the endoscope system in the related art that displays the direct-viewing observation image 111 and the side-viewing observation image 112.
In the second embodiment, the direct-viewing enlargement mode and the side-viewing enlargement mode are selected in accordance with the insertion/extraction of the insertion part 12a. However, in a case in which the movement detecting unit 201 detects the movement speed of the insertion part 12a (the tip part 12d) or the acceleration of movement of the insertion part 12a (the tip part 12d), a relative display ratio of the direct-viewing observation image 111 in the direct-viewing enlargement mode and the side-viewing observation image 112 in the side-viewing enlargement mode can be automatically set by additionally using the movement speed or acceleration of the insertion part 12a (the tip part 12d). For example, as shown in
On the other hand, in a case in which the movement speed of the insertion part 12a is negative and the insertion part 12a is moved in the extraction direction, the display ratio of the direct-viewing observation image 111 is reduced and the display ratio of the side-viewing observation image 112 is increased according to the movement speed of the insertion part 12a. That is, in a case in which the movement speed of the insertion part 12a is negative, a display mode is the side-viewing enlargement mode and the display control section 98 generates the display image 126 in which the side-viewing observation image is enlarged so as to have a display ratio corresponding to the movement speed and displays the display image 126 on the monitor 18. In the range equal to or smaller than a movement speed V2 (V2<0) at which the display ratio of the side-viewing observation image 112 reaches 100%, the display control section 98 generates the display image 129 in which substantially only the side-viewing observation image 112 is displayed and displays the display image 129 on the monitor 18.
In a case in which the insertion part 12a is not inserted and extracted, and the movement speed of the insertion part 12a is “0”, a display mode is automatically set to the standard mode. For this reason, in a case in which the movement speed of the insertion part 12a is “0”, the display control section 98 generates the display image 113 in the standard mode and displays the display image 113 on the monitor 18.
Considering not only the insertion/extraction of the insertion part 12a but also the movement speed (or acceleration) of the insertion part 12a during the insertion/extraction of the insertion part 12a as described above, the direct-viewing observation image 111 or the side-viewing observation image 112 can be appropriately enlarged and displayed in accordance with the more minute movement of the insertion part 12a. For this reason, even in a case in which the tip part 12d is moved by the movement of the insertion part 12a to more easily observe a portion to be noted during observation for diagnosis or the like in addition to an operation, which is to be performed before and after diagnosis or the like, such as the insertion or extraction of the insertion part 12a, the direct-viewing observation image 111 or the side-viewing observation image 112 can be enlarged and displayed in accordance with the movement of the tip part 12d.
The reason why the tip part 12d is moved in the insertion direction is that a portion to be noted is present in the tip direction, and the reason why the tip part 12d is moved in the extraction direction is that a portion to be noted is present in the lateral direction. Accordingly, in a case in which the direct-viewing enlargement mode or the side-viewing enlargement mode is automatically set as described above, an image, which includes a portion to be noted, of the direct-viewing observation image 111 and the side-viewing observation image 112 can be automatically enlarged and displayed. Further, in a case in which the insertion part 12a is quickly moved since a portion to be noted is present at a position distant from the current position of the tip part 12d, the direct-viewing observation image 111 or the side-viewing observation image 112 is enlarged so as to have a larger size. Accordingly, there is also an advantage that it is difficult to lose sight of a portion to be noted even though the insertion part 12a is quickly moved.
In the modification example, a display mode is switched to the direct-viewing enlargement mode and the side-viewing enlargement mode due to the insertion/extraction of the insertion part 12a and the movement speed of the insertion part 12a is additionally used to switch a display mode. However, a display mode may be switched due to the movement speed of the insertion part 12a regardless of the moving direction of the insertion part 12a. For example, as shown in
In the second embodiment, the direct-viewing enlargement mode and the side-viewing enlargement mode are selected in accordance with the insertion/extraction of the insertion part 12a. However, in a case in which the movement detecting unit 201 detects the orientation of the tip part 12d caused by a bend of the bendable part 12c, the display control section 98 can automatically select the direct-viewing enlargement mode and the side-viewing enlargement mode in accordance with the orientation of the tip part 12d caused by a bend of the bendable part 12c. Specifically, for example, as shown in
In the above description, a display mode is automatically set to the side-viewing enlargement mode in a case in which the bendable part 12c is not bent and the tip part 12d faces the insertion direction, but a display mode may be automatically set to the standard mode in a case in which the bendable part 12c is not bent and the tip part 12d faces the insertion direction.
Further, in a case in which the direct-viewing enlargement mode, the standard mode, or the side-viewing enlargement mode is switched in accordance with the orientation of the tip part 12d and the movement detecting unit 201 detects the bending angle, the bending speed, or the acceleration of the bendable part 12c, a relative display ratio of the direct-viewing observation image 111 in the direct-viewing enlargement mode and the side-viewing observation image 112 in the side-viewing enlargement mode can be automatically set by using the bending angle, the bending speed, or the acceleration of the bendable part 12c. For example, as shown in
In a case in which the bending angle is larger than 90°, the tip part 12d faces the rear side. In this case, since a portion to be noted in diagnosis or the like is likely to be present at a portion where the tip part 12d faces, it is good for the display control section 98 to generate the display image 128 in which substantially only the direct-viewing observation image 111 is displayed and to display the display image 128 on the monitor 18 (see
Furthermore,
In the first and second embodiments, the first blind spot portion 114 is masked and the first blind spot portion 114 is also reduced or enlarged in a case in which the side-viewing observation image 112 is reduced or enlarged in the direct-viewing enlargement mode or the side-viewing enlargement mode. For this reason, in the first and second embodiments, a portion of the side-viewing observation image 112 at which an object to be observed is shown up (a portion of the side-viewing observation image 112 except for the first and second blind spot portions 114 and 115) and the first blind spot portion 114 keep a relatively constant relationship between the sizes thereof before and after the reduction or enlargement. However, the first blind spot portion 114 and the portion of the side-viewing observation image 112 at which an object to be observed is shown up may not keep a constant relationship between the sizes thereof.
For example, the first blind spot portion 114 can be reduced as necessary, so that the display area of the first blind spot portion 114 can be reduced. In this case, as shown in
Specifically, for example, the first blind spot portion 114 of the display image 113 in the standard mode is a range that has a center C0 and an angle θ1 as shown in
In this embodiment, the mask display ratio setting unit 301 sets the range of the first blind spot portion 114, which has a center C0, to an angle θ2, which is smaller than at least the angle θ1, in the side-viewing enlargement mode. In a case in which the mask display ratio setting unit 301 sets the range of the first blind spot portion 114 to the angle θ2 in this way, the display image generating unit 102 further extends or compresses the direct-viewing observation image 111 and the side-viewing observation image 112 in a circumferential direction after reducing or enlarging the direct-viewing observation image 111 and the side-viewing observation image 112 in the radial direction as described above. Specifically, the display image generating unit 102 compresses a range, which has the angle θ1 and in which the first blind spot portion 114 is present, of the direct-viewing observation image 111 and the side-viewing observation image 112 in the circumferential direction and extends other portions thereof in the circumferential direction. Accordingly, the display image generating unit 102 allows the first blind spot portion 114 to be within the angle θ2. As a result, the display image generating unit 102 generates a display image 302 in which a relative display ratio of the mask covering the first blind spot portion 114 is reduced with respect to the side-viewing observation image 112, and displays the display image 302 on the monitor 18.
For example, in comparison with the display image 126 (see
In a case in which the display image 302 is to be generated, the reason why portions, which are present within the angle θ1, of not only the side-viewing observation image 112 but also the direct-viewing observation image 111 are compressed in the circumferential direction and portions thereof, which are present outside the angle θ1, are extended in the circumferential direction is to prevent deviation from being generated in the image of an object to be observed at a boundary between the direct-viewing observation image 111 and the side-viewing observation image 112. Further, the reason why the display area of the first blind spot portion 114 is reduced by the compression of the image and the display area of the other portion at which an object to be observed is shown up is increased by the enlargement of the image in the side-viewing enlargement mode is that, in the side-viewing enlargement mode, the display area of the first blind spot portion 114 is likely to be increased on the monitor 18 and an object to be observed is likely to be not easily observed since the first blind spot portion 114 not contributing to diagnosis or the like is likely to stand out even though being masked.
In the third embodiment, as an example, the first blind spot portion 114 is reduced and the other portion at which an object to be observed is shown is enlarged in the side-viewing enlargement mode. However, in the direct-viewing enlargement mode or the standard mode, the first blind spot portion 114 may be reduced and the other portion at which an object to be observed is shown up may be enlarged. In addition, in the side-viewing enlargement mode, the direct-viewing enlargement mode, or the standard mode, the first blind spot portion 114 may be enlarged and the other portion at which an object to be observed is shown up may be reduced.
In the third embodiment, the first blind spot portion 114 is reduced but the second blind spot portion 115 can be reduced. In this case, the mask display ratio setting unit 301 sets whether to enlarge the outer periphery of the side-viewing observation image 112 in the radial direction. For example, as shown in
The non-display of the second blind spot portion 115 of the modification example can be performed in all the cases of the direct-viewing enlargement mode, the side-viewing enlargement mode, and the standard mode. Accordingly, the non-display of the second blind spot portion 115 of the modification example can be performed in one or two or more display modes selected from the direct-viewing enlargement mode, the side-viewing enlargement mode, and the standard mode. Further, the non-display of the second blind spot portion 115 of the modification example can be performed instead of the reduction or enlargement of the first blind spot portion 114 of the third embodiment or in addition to the reduction or enlargement of the first blind spot portion 114 of the third embodiment.
In addition, the second blind spot portion 115 is not displayed at all in the modification example, but the mask display ratio setting unit 301 can allow a part of the second blind spot portion 115 to remain in the display image 304 while reducing the display area of the second blind spot portion 115 by setting the display ratio of the second blind spot portion 115. In this case, as in the case of the reduction of the first blind spot portion 114 of the third embodiment, it is good that the reduction ratio of the display area of the second blind spot portion 115 is set in accordance with a relative display ratio of the direct-viewing observation image 111 and the side-viewing observation image 112. For example, in the side-viewing enlargement mode, the display area of the second blind spot portion 115 can be reduced in inverse proportion to the display ratio of the side-viewing observation image 112.
The angle θ1 corresponding to a range in which the first blind spot portion 114 is present is stored in the display control section 98 in advance, but it is preferable that the data of the angle θ1 is updatable. In calibration or the like, the value of the angle θ1 is adjusted on the basis of, for example, the visibility or the like of the nozzles 51 and 52 from the side-viewing observation unit 42. As a result, it is preferable that the size of a region masking as the first blind spot portion 114 can be adjusted. This applies to the first embodiment and the like.
In the first, second, and third embodiments, the direct-viewing observation image 111 and the side-viewing observation image 112 are displayed at the center of the display image (the display image 113 and the like). However, the display positions of the direct-viewing observation image 111 and the side-viewing observation image in each display image can be adjusted. In a case in which the display positions of the direct-viewing observation image 111 and the side-viewing observation image are to be adjusted, the display control section 98 is provided with a display position setting unit 401 as shown in
Particularly, in a case in which the processor device 16 is provided with the movement detecting unit 201 of the second embodiment, the display position setting unit 401 can adjust the display position of the direct-viewing observation image 111 or the side-viewing observation image 112 in accordance with the movement detected by the movement detecting unit 201. Specifically, in a case in which the tip part 12d faces the insertion direction in the standard mode as shown in
In a case in which the tip part 12d faces the right in the standard mode as shown in
Further, in a case in which the tip part 12d faces the left in the standard mode as shown in
In the fourth embodiment, the display position of the direct-viewing observation image 111 is adjusted in accordance with the movement of the tip part 12d detected by the movement detecting unit 201. However, in a case in which the monitor 18 is a head mount display 421 as shown in
Further, in a case in which a doctor or the like wearing the head mount display 421 faces the front (the front of a wearer, such as a doctor, set by the calibration of the head mount display 421) in the standard mode, the display image 113 displayed at the center of the direct-viewing observation image 111 is displayed on the head mount display 421 (see
In the fourth embodiment, for simplification, a case in which a display mode is the standard mode has been described by way of example. However, the display position of the direct-viewing observation image 111 or the side-viewing observation image 112 can be adjusted even in the direct-viewing enlargement mode or the side-viewing enlargement mode. That is, the fourth embodiment can be arbitrarily combined with any one or more of the first, second, and third embodiments.
There is a case where other blind spots are present in the endoscopes 12 of each embodiment, such as the first embodiment, and the modification example thereof (hereinafter, referred to as the first embodiment and the like) in addition to a blind spot caused by the presence of the second protruding portion 32 and a blind spot formed outside the field of view of the side-viewing observation unit 42. Specifically, there is a case where a field 501 of view of the direct-viewing observation unit 41 and fields 502 of view of the side-viewing observation unit 42 do not overlap each other due to the performance of the image pickup lens 61 as shown in
For example, as shown in
The display control section 98 may display a message, which indicates the presence/absence of the blind spots 503, on the monitor 18 instead of the schematic diagram 505. Further, in a case in which the blind spots 503 are present between the field 501 of view of the direct-viewing observation unit 41 and the fields 502 of view of the side-viewing observation unit 42, the display control section 98 may generate a display image 523 in which the direct-viewing observation image 111 and the side-viewing observation image 112 are not connected to each other and a portion (hereinafter, referred to as a third blind spot portion) 513 corresponding to the blind spots 503 is masked as shown in
On the contrary to the fifth embodiment, there is a case where the field 501 of view of the direct-viewing observation unit 41 and the fields 502 of view of the side-viewing observation unit 42 overlap each other in the endoscopes 12 of the first embodiment and the like due to the performance of the image pickup lens 61 as shown in
For this purpose, it is preferable that the display control section 98 is provided with, for example, an overlapping region-detecting unit 611 and an overlapping region-removing unit 612 as shown in
In a case in which the display control section 98 is provided with the overlapping region-detecting unit 611 as in the sixth embodiment, it is good that the display control section 98 is further provided with a signal level-correcting unit 613 as shown in
In a case in which the direct-viewing observation unit 41 and the side-viewing observation unit 42 have the overlapping fields 601 of view, the sixth embodiment and the modification example thereof can be arbitrarily combined with the direct-viewing enlargement mode, the side-viewing enlargement mode, or the standard mode of the first embodiment and the like.
In the first embodiment and the like, the display control section 98 connects the direct-viewing observation image 111 to the side-viewing observation image 112 by enlargement, reduction, or the like of the direct-viewing observation image 111 or the side-viewing observation image 112 in a case in which the display image 113 and the like are generated by the display image generating unit 102. However, there is a case where the state of aberration of the direct-viewing observation image 111 and the state of aberration of the side-viewing observation image 112 caused by the performance of the image pickup lens 61 are different from each other. If the direct-viewing observation image 111 and the side-viewing observation image 112 are connected to each other by only enlargement, reduction, or the like of the direct-viewing observation image 111 or the side-viewing observation image 112 in a case in which the state of aberration of the direct-viewing observation image 111 and the state of aberration of the side-viewing observation image 112 are different from each other, there is a case where an object to be observed is seen so as to be distorted in the display image 113 and the like due to a difference between the aberration of the direct-viewing observation image 111 and the aberration of the side-viewing observation image 112.
For example, the distortion 701 of the direct-viewing observation image 111 and the distortion 702 of the side-viewing observation image 112 are different from each other as shown in
The display control section 98 is provided with an aberration connecting unit 705 as shown in
Since the performance of the image pickup lens 61 is already known at the time of the manufacture of the endoscope 12, transformation processing to be performed on the direct-viewing observation image 111 or the side-viewing observation image 112 has been determined in advance in the seventh embodiment. For this reason, the aberration connecting unit 705 has stored parameters for transformation processing, which is to be performed on the direct-viewing observation image 111 or the side-viewing observation image 112, in advance. However, more precisely, in a case in which a distance between the tip part 12d of the endoscope 12 and an object to be observed is changed during observation, the state of aberration of the distortion 701 of the direct-viewing observation image 111 and the state of aberration of the distortion 702 of the side-viewing observation image 112 are changed according to the distance between the tip part 12d and the object to be observed. Accordingly, in a case in which the display control section 98 is provided with the aberration connecting unit 705, it is preferable that the endoscope 12 or the processor device 16 is provided with a distance measurement unit 706 as shown in
In a case in which blind spots 503 are present between the field 501 of view of the direct-viewing observation unit 41 and the fields 502 of view of the side-viewing observation unit 42, portions near boundaries between the field 501 of view of the direct-viewing observation unit 41 and the fields 502 of view of the side-viewing observation unit 42 are portions near boundaries between the field 501 of view of the direct-viewing observation unit 41 and the blind spots 503, portions near boundaries between the fields 502 of view of the side-viewing observation unit 42 and the blind spots 503, or arbitrary portions in the ranges of the blind spots 503. In a case in which the overlapping fields 601 of view are present in the field 501 of view of the direct-viewing observation unit 41 and the fields 502 of view of the side-viewing observation unit 42, portions near boundaries between the field 501 of view of the direct-viewing observation unit 41 and the fields 502 of view of the side-viewing observation unit 42 are portions near boundaries between the field 501 of view of the direct-viewing observation unit 41 and the overlapping fields 601 of view, portions near boundaries between the fields 502 of view of the side-viewing observation unit 42 and the overlapping fields 601 of view, or arbitrary portions in the ranges of the overlapping fields 601 of view. In a case in which blind spots 503 are not present between the field 501 of view of the direct-viewing observation unit 41 and the fields 502 of view of the side-viewing observation unit 42 and overlapping fields 601 of view are also not present, portions near boundaries between the field 501 of view of the direct-viewing observation unit 41 and the fields 502 of view of the side-viewing observation unit 42 are boundaries between the field 501 of view of the direct-viewing observation unit 41 and the fields 502 of view of the side-viewing observation unit 42 or portions near the boundaries.
In the first embodiment and the like, the control unit 96 performs AE control but there is a case where the brightness of the direct-viewing observation image 111 and the brightness of the side-viewing observation image 112 are different from each other depending on the distance between the tip part 12d and an object to be observed or the performance of the direct-viewing illumination units 54 and 81 and the side-viewing illumination unit 43. In a case in which the brightness of the direct-viewing observation image 111 and the brightness of the side-viewing observation image 112 are different from each other, it is difficult for the control unit 96 to perform AE control for allowing both the direct-viewing observation image 111 and the side-viewing observation image 112 to always have common constant brightness. For this reason, as shown in
In the eighth embodiment, the control unit 96 determines any one of the exposures calculated using the direct-viewing observation image 111 and the exposure calculated using the side-viewing observation image 112 as an exposure for AE control to be actually performed. However, instead, the control unit 96 can determine an average value of the exposure calculated using the direct-viewing observation image 111 and the exposure calculated using the side-viewing observation image 112 as an exposure for AE control to be actually performed. Further, an exposure for AE control, which is to be actually performed, may be calculated, by setting, using the exposure calculated using the direct-viewing observation image 111 and the exposure calculated using the side-viewing observation image 112. For example, the exposure calculated using the direct-viewing observation image 111 and the exposure calculated using the side-viewing observation image 112 are weighted and averaged, and the result of the weighting and averaging can be employed as an exposure for AE control to be actually performed.
In a case in which a direct-viewing observation image 111 and a side-viewing observation image 112 are to be obtained at respective different timings, an exposure to be used to obtain the direct-viewing observation image 111 can be calculated using the direct-viewing observation image 111 and an exposure to be used to obtain the side-viewing observation image 112 can be calculated using the side-viewing observation image 112.
For example, in a case in which the control unit 96 performs AE control by adjusting the amount of illumination light, as shown in
Since an exposure to be used to obtain a direct-viewing observation image 111 is calculated using the direct-viewing observation image 111 and an exposure to be used to obtain a side-viewing observation image 112 is calculated using the side-viewing observation image 112 in a case in which the direct-viewing observation image 111 and the side-viewing observation image 112 are to be obtained at respective different timings as described above, both the direct-viewing observation image 111 and the side-viewing observation image 112 can be acquired with appropriate exposure.
The AE control of the eighth embodiment and the modification example thereof can be arbitrarily combined with the first embodiment and the like.
In a case and the like in which a direct-viewing observation image 111 and a side-viewing observation image 112 are to be obtained at respective different timings, a direct-viewing observation image 111 and a side-viewing observation image 112 having properties different from each other can be obtained. For example, one of the direct-viewing observation image 111 and the side-viewing observation image 112 can be converted into a normal image, which is obtained when an image of an object to be observed is picked up with the use of white light as illumination light and expresses the object to be observed with natural color tones, and the other of the direct-viewing observation image 111 and the side-viewing observation image 112 can be converted into an image (hereinafter, referred to as a special image) that has been subjected to processing and the like so as to have an aspect different from the normal image. Examples of the special image include: an image in which a specific tissue or structure, such as a blood vessel, is enhanced by frequency enhancement processing, contour enhancement processing, marking processing (processing for displaying the position of a lesion or the like by applying a frame or the like to the lesion or the like), or the like; an image in which a specific tissue or structure, such as a blood vessel, is more easily shown up than in a normal image with the use of illumination light having a very narrow wavelength band or the selection of the wavelength or the wavelength band of illumination light or the combinations of illumination light (for example, a so-called narrow-band light observation image); an image in which a specific lesion and the like can be distinguished from peripheral tissues and the like through color differences by color tone conversion processing or the like (for example, color tone conversion using a tone curve, color tone conversion using remapping in a specific color space, or the like); an image that is obtained by calculating biological information about an object to be observed and coloring a normal image with the use of the calculated biological information (for example, an oxygen saturation image expressing a value of oxygen saturation by a color); and the like.
In a case in which one of the direct-viewing observation image 111 and the side-viewing observation image 112 is converted into a normal image and the other of the direct-viewing observation image 111 and the side-viewing observation image 112 is converted into a special image as described above, the processor device 16 is provided with an image processing unit 830 as shown in
In a case in which at least one of the direct-viewing observation image 111 and the side-viewing observation image 112 is converted into a special image, the discovery of a lesion and the like can be supported in comparison with a case in which each of both the direct-viewing observation image 111 and the side-viewing observation image 112 is converted into a normal image. Further, there is also an advantage of providing information, such as biological information, in a case in which an image representing biological information is to be generated.
In a case in which only one of the direct-viewing observation image 111 and the side-viewing observation image 112 is converted into a special image, it is preferable that the direct-viewing observation image 111 is converted into a normal image and the side-viewing observation image 112 is converted into a special image. The side-viewing observation image 112 is often used for screening for discovering a lesion and the like, and is likely to be an image allowing a lesion and the like to not be better identified than the direct-viewing observation image 111. Accordingly, in a case in which the side-viewing observation image 112 is converted into a special image, screening can be supported and the accuracy of the screening can be improved.
In the ninth embodiment, the direct-viewing observation image 111 and the side-viewing observation image 112 are acquired (picked up) at respective different timings. However, the direct-viewing observation image 111 and the side-viewing observation image 112 can be simultaneously acquired and at least one of the direct-viewing observation image 111 and the side-viewing observation image 112 can be converted into a special image. For example, an image of an object to be observed is picked up with the use of white light as illumination light, and the direct-viewing observation image 111 and the side-viewing observation image 112 are simultaneously acquired. Then, normal image processing (image processing for transforming an image into a normal image) is performed on the direct-viewing observation image 111 to convert the direct-viewing observation image 111 into a normal image. On the other hand, color conversion processing for enhancing a lesion is performed on the side-viewing observation image 112, which is obtained simultaneously with the direct-viewing observation image 111, to convert the side-viewing observation image 112 into a special image. In a case in which a special image is to be generated by image processing without requiring switching illumination light as described above, at least one of the direct-viewing observation image 111 and the side-viewing observation image 112 can be converted into a special image even though the direct-viewing observation image 111 and the side-viewing observation image 112 are simultaneously acquired.
Further, in a case in which the direct-viewing illumination units 54 and 81 and the side-viewing illumination unit 43 are adapted to be independent of each other, different kinds of illumination light can be simultaneously applied to the field 501 of view of the direct-viewing observation unit 41 and the fields 502 of view of the side-viewing observation unit 42. In this case, even though the switching of illumination light is required for the generation of a special image, the direct-viewing observation image 111 and the side-viewing observation image 112 are simultaneously acquired and at least one of the direct-viewing observation image 111 and the side-viewing observation image 112 can be converted into a special image.
In the ninth embodiment, at least one of the direct-viewing observation image 111 and the side-viewing observation image 112 is converted into a special image. However, instead, the brightness of one of the direct-viewing observation image 111 and the side-viewing observation image 112 may be made higher or lower than the brightness of the other thereof. For example, the side-viewing observation image 112 can be made brighter than the direct-viewing observation image 111. A method of allowing the side-viewing observation image 112 to be brighter than the direct-viewing observation image 111 and a method of allowing the direct-viewing observation image 111 to be darker than the side-viewing observation image 112 are employed as a method of allowing the side-viewing observation image 112 to be brighter than the direct-viewing observation image 111. The control unit 96 adjusts a relative balance between the amount of illumination light for direct viewing and the amount of illumination light for side viewing in the modification example (see
In the first embodiment and the like, the display regions and the display positions of the direct-viewing observation image 111 and the side-viewing observation image 112 of the display image 113 and the like are determined in advance. Particularly, in the fourth embodiment and the modification example thereof, the display position of the direct-viewing observation image 111 or the side-viewing observation image 112 has been adjusted in accordance with the orientation of the tip part 12d, or the like but the display regions of the direct-viewing observation image 111 and the side-viewing observation image 112 of the display image 113 and the like are determined in advance. However, the display regions of the direct-viewing observation image 111 and the side-viewing observation image 112 can be arbitrarily determined.
The display regions of the direct-viewing observation image 111 and the side-viewing observation image 112 are the ranges of the direct-viewing observation image 111 and the side-viewing observation image 112 that are displayed on the monitor 18 by the display image 113 and the like. For example, since the centers C0 of the direct-viewing observation image 111 and the side-viewing observation image 112 are disposed at the center of the display image 113 in the display image 113, a part of the side-viewing observation image 112 originally having an annular shape protrudes to the outside of the display image 113 and is not displayed in the display image 113. In contrast, the range of the side-viewing observation image 112, which is used in the display image 113, is the display region of the side-viewing observation image 112.
In a case in which the display ranges of the direct-viewing observation image 111 and the side-viewing observation image 112 are to be adjusted in the display image 113 and the like, the display control section 98 is provided with a display region setting unit 1001 as shown in
For example, in a case in which the display region setting unit 1001 sets an interrelationship between the direct-viewing observation image 111 and the side-viewing observation image 112 and the display region 1002 as shown in
In the tenth embodiment, the display control section 98 allows the entire first blind spot portion 114, which corresponds to a blind spot in which the second protruding portion 32 is shown up, not to be displayed by offsetting the display positions of the direct-viewing observation image 111 and the side-viewing observation image 112 with respect to the display region 1002 on the monitor 18 serving as the display unit. For this reason, since the first blind spot portion 114 can be excluded from the display on the monitor 18, a display focusing on the observation of an object to be observed can be made without concern about the presence of the first blind spot portion 114 in the display on the monitor 18.
The entire first blind spot portion 114 is not displayed in the tenth embodiment. However, in a case in which the display control section 98 is provided with the display region setting unit 1001, it is preferable that the display control section 98 allows at least a part of a blind spot not to be displayed by offsetting the display positions of the direct-viewing observation image 111 and the side-viewing observation image 112 with respect to the display region 1002 on the monitor 18 serving as the display unit. The display control section 98 can allow at least a part of the first blind spot portion 114, which corresponds to a blind spot in which the second protruding portion 32 is shown up, not to be displayed by enlarging the direct-viewing observation image 111 and the side-viewing observation image 112 with respect to the display region 1002 on the monitor 18 serving as the display unit after offsetting the display positions of the direct-viewing observation image 111 and the side-viewing observation image 112 with respect to the display region 1002, instead of offsetting the display positions of the direct-viewing observation image 111 and the side-viewing observation image 112 with respect to the display region 1002 as described above. The reason for this is that a user focuses more on the observation of an object to be observed than in a case in which the entire first blind spot portion 114 of the display image 113 and the like is made to be within the display region 1002 in a case in which at least a part of the first blind spot portion 114 is not displayed. For example, in a case in which the display control section 98 enlarges the direct-viewing observation image 111 and the side-viewing observation image 112 with respect to the display region 1002 as shown in
Further, in a case in which the display control section 98 enlarges the direct-viewing observation image 111 and the side-viewing observation image 112 with respect to the display region 1002 as shown in
It is preferable that the non-display of the first blind spot portion 114 and the second blind spot portion 115, which is achieved by the offsetting or enlargement, can be selected as a display mode. That is, it is preferable that the display control section 98 includes a blind spot-non-display mode in addition to the standard mode, the direct-viewing enlargement mode, or the side-viewing enlargement mode of the first embodiment and the like or instead of the standard mode, the direct-viewing enlargement mode, or the side-viewing enlargement mode of the first embodiment and the like. In a case in which a display mode is set to the blind spot-non-display mode, the display control section 98 allows at least a part of the first blind spot portion 114 and the second blind spot portion 115, which corresponds to a blind spot in which the second protruding portion 32 is shown up, not to be displayed and displays the direct-viewing observation image 111 and the side-viewing observation image on the monitor 18 serving as the display unit.
The display control section 98 can change the ratio of a portion, which is not to be displayed, of the first blind spot portion 114 that corresponds to a blind spot in which the second protruding portion 32 is shown up. For example, in a case in which the processor device 16 is provided with the movement detecting unit 201 as shown in
More specifically, in a case in which the movement detecting unit 201 is to detect the insertion/extraction of the insertion part 12a, the display region setting unit 1001 can set the ratio of a portion, which is not to be displayed, of the first blind spot portion 114 or the second blind spot portion 115 in accordance with the speed (or acceleration) of insertion/extraction of the insertion part 12a as shown in
In
The tenth embodiment can be arbitrarily combined with the first embodiment and the like. For example, the display control section 98 can adjust the display ratios of the side-viewing observation image 112 and the direct-viewing observation image 111 in the same manner as the first embodiment and the like, and can change the ratio of a portion, which is not to be displayed, of the first blind spot portion 114 or the second blind spot portion 115 in accordance with the display ratios of the side-viewing observation image 112 and the direct-viewing observation image 111. This is the same as a case in which the first blind spot portion 114 is reduced in accordance with the display ratios of the side-viewing observation image 112 and the direct-viewing observation image 111 in the third embodiment.
In a case in which the display control section 98 allows the first blind spot portion 114 or the second blind spot portion 115 not to be displayed as in the tenth embodiment (particularly, in a case in which the display control section 98 includes the blind spot-non-display mode), it is preferable that the nozzles 51 and 52, which eject washing liquid to the side-viewing observation unit 42 to wash the side-viewing observation unit 42, are provided in a non-display region (a region outside the display region 1002) that is not to be displayed in a case in which the display control section 98 displays the side-viewing observation image 112 on the monitor 18 serving as the display unit. In this case, since the nozzles 51 and 52 do not need to be necessarily provided at the second protruding portion 32, there is a case where the washability of the side-viewing observation unit 42 is improved depending on the arrangement of the nozzles 51 and 52.
The endoscopes 12 of the first embodiment and the like are inserted into an object to be observed in a state in which a complicated shape in which the first and second protruding portions 31 and 32 protrude from the tip surface 21 of the tip part 12d is exposed to the outside. However, the endoscope 12, which can perform direct-viewing observation and side-viewing observation, can be used in a state in which the tip part 12d (particularly, a portion corresponding to the first and second protruding portions 31 and 32) is covered with a tip cap 1101 as shown in, for example,
Since the tip cap 1101 is transparent, the tip cap 1101 basically does not hinder the direct-viewing observation using the direct-viewing illumination unit 54, the direct-viewing illumination unit 81, and the direct-viewing observation unit 41 and the side-viewing observation using the side-viewing illumination unit 43 and the side-viewing observation unit 42. However, although the tip cap 1101 is transparent, there is a case where the image of an object to be observed is distorted, for example, at an outline 1101A of the tip cap 1101 or near the outline 1101A. For this reason, it is preferable that the display control section 98 generates a display image 1120 in which a line 1111 (for example, a colored line or band) or the like representing the presence of the outline 1101A of the tip cap 1101 is displayed as shown in
The display control section 98 may generate a display image 1121 in which the outline 1101A of the tip cap 1101 or a portion 1122 of the tip cap 1101 near the outline 1101A, at which distortion is likely to be displayed in the image of the object to be observed, are masked as shown in
The endoscope 12 is used in a state in which an auxiliary member 1201, which pushes and smoothes out the creases of a lumen (a large intestine or the like) serving as an object to be observed, is mounted on the tip part 12d in addition to the tip cap 1101 as shown in
Further, there is also a case where the auxiliary member 1201 includes claw-shaped members (hereinafter, referred to as claw members) 1207 locking the auxiliary member 1201 to the tip part 12d instead of the locking member 1206 as shown in
In the first embodiment and the like, the second protruding portion 32 forms a blind spot in the side-viewing observation unit 42. However, as shown in
In the first embodiment and the like, an object to be observed is substantially fully displayed in the display image 113 and the like. However, there have also been many cases where the display screen of a monitor 18 in recent years is laterally long or vertically long. Even though the direct-viewing observation image 111 and the side-viewing observation image 112 are appropriately displayed in a laterally long display region as in a display image 1401 shown in
In a case in which the marginal region 1402 is formed in a display screen due to an aspect ratio of the display screen of the monitor 18 or the like as described above, it is preferable that the display control section 98 provides an information display region 1404 in the marginal region 1402 as in a display image 1403 shown in
A focusing angle of view has been displayed above in the information display region 1404 by way of example, but other information may be displayed in the information display region 1404. That is, as long as the marginal region 1402 is provided with the information display region 1404 and some sort of information is displayed in the information display region 1404 so that at least a part of the marginal region 1402 can be effectively used, the contents of information to be displayed is arbitrary. For example, as in a display image 1411 shown in
In addition, in a case in which the first blind spot portion 114 is present even though the marginal region 1402 is not present, an information display region 1502 is provided in the first blind spot portion 114 and information, such as the model name of the endoscope 12, is displayed in the information display region 1502 as in a display image 1501 shown in
In the first embodiment and the like, the display image 113 and the like are generated using the direct-viewing observation image 111 and the side-viewing observation image 112, and are displayed on the monitor 18. However, a three-dimensional model (3D model) 1605 can be generated using the direct-viewing observation image 111 or the side-viewing observation image 112. In this case, for example, the processor device 16 may be provided with a 3D modeling unit 1601 and a storage unit 1602 as shown in
The endoscope system 10, which is used in a state in which the insertion part 12a is inserted into a subject, has been described in the first embodiment and the like by way of example, but the first embodiment and the like can also be used for a capsule endoscope that is used in a state in which an examinee swallows the capsule endoscope.
The direct-viewing observation unit 41 and the side-viewing observation unit 42 share the image pickup lens 61 and the image sensor 66 in the first embodiment and the like, but the image pickup lens 61 and the image sensor 66 of the direct-viewing observation unit 41 may be separate from the image pickup lens 61 and the image sensor 66 of the side-viewing observation unit 42.
Number | Date | Country | Kind |
---|---|---|---|
2016-197542 | Oct 2016 | JP | national |
Number | Name | Date | Kind |
---|---|---|---|
9526402 | Honda | Dec 2016 | B2 |
9554097 | Sasaki et al. | Jan 2017 | B2 |
20050038317 | Ratnakar | Feb 2005 | A1 |
20050073578 | Odlivak | Apr 2005 | A1 |
20080234866 | Kishi | Sep 2008 | A1 |
20110282148 | Kase | Nov 2011 | A1 |
20120088969 | Takahira | Apr 2012 | A1 |
20120188351 | Kaku | Jul 2012 | A1 |
20130076879 | On | Mar 2013 | A1 |
20140046131 | Morita | Feb 2014 | A1 |
20140204187 | Sasaki | Jul 2014 | A1 |
20150187119 | Masumoto | Jul 2015 | A1 |
20150272423 | Ito | Oct 2015 | A1 |
20150282691 | Amling | Oct 2015 | A1 |
20160338575 | Honda et al. | Nov 2016 | A1 |
20170085762 | Obara et al. | Mar 2017 | A1 |
Number | Date | Country |
---|---|---|
2762059 | Aug 2014 | EP |
2012-245157 | Dec 2012 | JP |
2013066646 | Apr 2013 | JP |
2016-007745 | Jan 2016 | JP |
2011055641 | May 2011 | WO |
2014088076 | Jun 2014 | WO |
2015122355 | Aug 2015 | WO |
2015146836 | Oct 2015 | WO |
2016072237 | May 2016 | WO |
Entry |
---|
“Search Report of European Counterpart Application,” dated Mar. 1, 2018, pp. 1-8. |
“Office Action of Japan Counterpart Application”, dated Apr. 28, 2020, with English translation thereof, p. 1-p. 4. |
“Office Action of Japan Counterpart Application,” with English translation thereof, dated Dec. 24, 2019, p. 1-p. 6. |
Number | Date | Country | |
---|---|---|---|
20180092515 A1 | Apr 2018 | US |