The invention relates to an endoscope system as well as to a supplementary module for an endoscope, and to endoscope optics for measuring and/or imaging three-dimensional structures.
Endoscopes are used for carrying out visual examinations of human, animal and technical cavities, with which a three-dimensional observation of objects is possible. Video endoscopes or video cameras which are used together with an endoscope, in contrast, only permit a two-dimensional observation. It is desirable with many applications to not only to be able to see a two-dimensional image but, for example, also to be able to represent three-dimensional structures of an object to be examined.
It is the object of the invention to provide an endoscope or endoscope system, with whose help structures may be detected and imaged in a three-dimensional manner. It is particularly desirable to provide a system for this, which may be integrated into conventional endoscopes or may be supplemented with the existing endoscope.
This object is achieved by an endoscope system for detecting three-dimensional structures, the system having at least one radiation source, which emits modulated radiation and is applied in a manner such that an object to be examined by way of the endoscope system may be illuminated with the radiation, at least one sensor for detecting the radiation reflected by the object, and an evaluation device, which obtains readings of the detected radiation from the sensor, determines phase differences at least one picture point on the basis of these readings, and computes the distance of the object to the sensor on the basis of these phase differences
This object is further achieved by a supplementary module for an endoscope, which comprises an interface for connection to an endoscope, wherein the supplementary module comprises a picture observation device and a radiation source, which emits a modulated radiation which may be coupled into the endoscope via the interface, and a sensor, wherein the sensor, via the interface, may detect the modulated radiation which was reflected by an object to be examined by the endoscope, and is designed for detecting the radiation of at least one picture point, which is reflected by the object, and the sensor is connected to an evaluation device which obtains readings of the detected radiation from the sensor, determines phase differences at least one picture point on the basis of these readings, and computes the distance of the object from the sensor on the basis of these phase differences.
This object is further achieved by endoscope optics designed for application in an endoscope and which is equipped with a measurement system and comprises the following components: a radiation source, which emits a modulated radiation and may be connected to the endoscope in a manner such that an object to be examined by way of the endoscope may be illuminated with the radiation, a sensor for detecting the radiation reflected by the object at least one picture point, and an evaluation device which obtains readings of the detected radiation from the sensor, determines phase differences at the several picture points on the basis of these readings, and computes the distance of the object to the sensor on the basis of these phase differences.
The endoscope system according to one embodiment of the invention, apart from the usual components of an endoscope, comprises a radiation (beam) source which emits modulated radiation. This radiation source is arranged in the endoscope system in a manner such that an object to be examined may be illuminated with the radiation. For this, the radiation source is either arranged at the distal end of the endoscope, or however further proximally or outside the endoscope, wherein in the latter cases, a fiber-optic must be provided for leading the radiation to the distal end of the endoscope. In any case, the radiation exits at the distal end of the endoscope and radiates the object to be examined or a region of the object to be examined.
According to another embodiment of the invention, the endoscope system further comprises a sensor with one or preferably several sensor elements. The sensor or the sensor elements serve for detecting the modulated radiation which is reflected by the object. Thereby, one envisages detecting the reflected radiation at different picture points, in particular by way of several sensor elements. These correspond to different points of the object.
According to a further embodiment of the invention, an evaluation device is provided, which receives the readings generated by the sensor on account of the detected, reflected radiation, and computes the distance of the object to the sensor on the basis of these readings. Thereby, it is particularly the points of the object which correspond to individual picture points which are measured, i.e. the distance of these points to the sensor is determined. The distance measurement is effected in the evaluation device, in a manner such that in particular, with the knowledge of the modulation of the radiation source, phase differences in the radiation are determined from the readings at the individual sensor elements. One may determine the signal transfer time between the radiation source and the individual sensor elements from these phase differences. The distance may then be determined from this transfer time.
The evaluation device according to the invention, by way of the distance measurement, permits an object to be observed or examined, to be measured with regard to its three-dimensional scope. Here, preferably a certain number of picture points are detected, so that a three-dimensional image of the object may be produced.
The measurement system according to the invention may be integrated into an endoscope or an endoscope system in different manners. It is thus possible to arrange the radiation source as well as the sensor for detecting the reflected radiation, at the proximal end of the endoscope, or to arrange it connected to the proximal end, and to lead the modulated radiation to the distal end, and the reflected radiation to the proximal end via fiber-optics or optical systems in the endoscope. Alternatively, it is possible to arrange the radiation source and/or the sensor directly at the distal end of the endoscope, or in the inside of the endoscope between the proximal and the distal end. The evaluation device is preferably provided as an external component, which is connected to the radiation source and the sensor via cable. However, it is conceivable to integrate the evaluation device directly into the endoscope or into a module applied directly onto the endoscope.
The readings which the evaluation device receives from the sensor or the sensor element for the individual picture points, preferably include the radiation intensity measured at the individual picture points, in dependence on time. Here, a continuous detection and evaluation of the intensity does not need to be effected, but rather it may be sufficient to predetermine the intensity of the radiation at the individual picture points and to determine consecutive points in time, so that the phase course of the radiation may be determined at the picture points. Then, with the knowledge of the phase course or the modulation at the radiation source, one may determine the phase differences, on the basis of which one then determines the signal transfer times at the individual picture points, or in particular the differences of the transfer times between the signals at the several picture points, in order to determine the three-dimensional structure of the object.
The detection of the radiation intensity is effected according to the state of the art in a manner such that the sensor and the evaluation device are designed as a TOF-camera (time of flight), as is known, for example, from German Patent DE 44 40 613 C1 or European Patent Application Publication EP 1 622 200 A1.
The evaluation device is designed for the computation and output of the computed distance values and, as the case may be, the output of a three dimensional representation of the object. The results of the distance measurement may be selectively displayed e.g. on a monitor and the object may be represented in its three-dimensional structure for the user, additionally on a second monitor.
The endoscope system preferably comprises a common endoscope preferably a video endoscope for the transmission and representation of a picture of the object to be examined. This endoscope permits the user to observe the object to be examined (preferably continuously) on a monitor, during the measurement of this object, which may be particularly advantageous with medical applications.
Further preferably, the endoscope system comprises an endoscope with at least one optical system extending from the distal to the proximal end, which serves for the picture transmission as well as for the transmission of the modulated radiation reflected by the object. This means that the optical system on the one hand serves for transmitting a picture of the object to be examined, from the distal end of the endoscope to the proximal end of the endoscope, where for example a video camera or an eyepiece may be arranged for visual observation of the object, in the known manner. According to an embodiment of the invention, the optical system is however on the other hand also applied for leading the modulated radiation reflected by the object, from the distal end to the proximal end of the endoscope. Then, the sensor for detecting and evaluating the reflected radiation may be arranged at the proximal end.
The endoscope system further preferably comprises an endoscope with a fiber-optic which is designed for transmitting light from a light source which is arranged proximally or externally, to the distal end, wherein the light is provided for the illumination of an object to be observed by the endoscope optics. Moreover, a corresponding light source may be arranged in or at the distal end of the endoscope. The light exits at the distal end of the endoscope and illuminates the region which may be observed by the endoscope optics or the optical system.
The fiber-optic may be additional particularly preferably used for transmitting the modulated radiation from the proximal to the distal end of the endoscope, wherein the radiation then exits at the distal end. This design permits the modulated radiation to be led through the endoscope or through the endoscope shank preferably without additional elements, when the radiation source is not arranged directly at the distal end of the endoscope.
The illumination of three-dimensional structures to be observed and represented in a visual manner, is preferably effected by way of a radiation source or light source, which emits modulated light preferably in the visible range. This permits the use of the optics or optical systems and fiber-optics in conventional endoscopes, for the transmission of the modulated radiation. Thus in particular, it is possible for the measurement system according to embodiments of the invention for the three-dimensional detection of objects, to be integrated into conventional endoscopes, or to be retrofitted with conventional endoscopes.
This modulated radiation source is furthermore designed for emitting pulsed light. I.e. the light emitted by the radiation source is pulsed in a predefined clock pulse. The modulated light may for example be switched in an inactive manner for those time durations in which the video sensor is used, and thus counteract undesired influences on the color perception of the system.
For this, a clock generator is advantageously provided for the control of the radiation source and of the sensor and the evaluation device. The clock generator on the one hand sets the modulation and the pulse frequency of the emitted light. On the other hand, the clock generator also sets the clock pulse for the detection of the reflected radiation at the sensor and the subsequent evaluation. Thus, for detecting phase differences for the evaluation device, it is necessary to know the initial phase course of the radiation emitted by the radiation source, in order to then be able to determine the phase differences from the reading of the sensor or the sensor elements.
According to a preferred embodiment, common optics for transmitting a picture of the object to be examined, as well as the modulated radiation reflected by the object, are provided in the endoscope, wherein a separation device or switch-over device is arranged at the exit side of the common optics, which leads the reflected modulated radiation to the sensor, and the picture to an eyepiece or to a video sensor. The separating device or switch-over device thus serves for separating the normal picture from the reflected, modulated radiation, so that both signals may be supplied to different detection systems, specifically the modulated radiation to the sensor, and the normal picture signals to a video sensor or to an eyepiece for direct observation.
Alternatively, it is possible to provide a switch-over device which merely for the measurement by way of the sensor, briefly leads the picture or the radiation from the common optics to the sensor. This switch-over device in this context is preferably controlled by the clock generator of the radiation source. In this case, the normal picture which is visible at an eyepiece or is detected by a video sensor, is interrupted briefly for the moment of measurement. The measurement may advantageously however be effected via the sensor in a rapid manner such that this interruption may not be noticed by the human eye, and a quasi continuous picture may be provided for the observer.
According to a further preferred embodiment, the sensor according to the invention may be part of a video sensor for picture detection, or may form a construction unit with this. This means that the sensor may be integrated into a normal video sensor, which serves for the detection of the picture transmitted by the endoscope optics in the known manner. For this, memory cells with switches may be arranged for example at individual picture points, particularly preferably at all picture points such as with a TOF-camera, which permit the intensities detected by the individual sensor cells or sensor elements to be stored in the memory cells and subsequently read out for further computation. One may make do with a part device or deflection device with such a design, and it is very simply possible to integrate the measurement system according to an embodiment of the invention into the normal picture transmission or picture processing of an endoscope with the help of video technology.
A further possible embodiment of the invention envisages the sensor device for detecting the reflected modulated radiation, as well as optics or a video sensor for picture observation being integrated in a common module, which via an interface is applied on the endoscope shank on the proximal side. Thereby, the interface may be designed in a manner such that it is possible to attach the module to convention endoscopes or endoscope shanks at the proximal side, so that the measurement system may be integrated very easily into conventional endoscopes. Then ideally, only the video picture detection system attached to the endoscope at the proximal side needs to be replaced by the described module, which integrates the picture detection as well as the described measurement technology for three-dimensional detection of the object.
According to an embodiment of the invention, an illumination connection is further preferably provided, which is connected to a light source for illuminating the object and to the radiation source which emits the modulated radiation. The illumination connection may, for example at the exit side, be designed as a connection plug which is connected to the fiber-optic cable connection on the endoscope shank. This means that the illumination connection serves for coupling the radiation or the light from the radiation source into the illumination path of the endoscope, which transmits the light to the distal end of the endoscope for illuminating the object. The coupling-in location lies in the described illumination connection, which on the entry side may be connected to two radiation sources or light sources, specifically to the light source for illumination, and to the described radiation source for modulated radiation. Only one exit is provided at the exit side, which is preferably designed as a standard connection plug or standard connection coupling, which may be connected to the usual fiber-optic connections of endoscope shanks. In this manner, it is very simply possible to also use conventional endoscopes for the measurement system according to the invention, and to couple the required modulated radiation into conventional endoscopes.
As already mentioned in the preceding description, it is particularly preferable to be able to retrofit existing or conventional endoscopes in a manner such that a three-dimensional detection or measurement of the object to be examined is possible. According to an embodiment of the invention, a supplementary module for an endoscope is provided for this, which as an accessory part may be attached to endoscopes or be integrated into endoscope systems.
For this, the supplementary module comprises an interface, via which it may be attached to an endoscope and connected to this. The interface for this is preferably designed as a standard interface, in order to be able to be connected to conventional endoscope types or types of endoscope optics. A picture observation device is arranged in the supplementary module and the picture of the object region which is situated in the field of viewing of the endoscope may be detected via this device in the conventional way. For this, one may provide an eyepiece or a video sensor with a video system connecting thereto. Furthermore, a connection for an external light source is preferably provided in the supplementary module, and this light source effects the usual illumination of the region of the endoscope on the distal side. For this, the light is led from the light source, e.g. through the supplementary module to a connection plug, and via this into the fiber-optic of the endoscope. The light source may alternatively be also integrated into the supplementary module.
According to an embodiment of the invention a radiation source, particularly preferably, again a light source which emits modulated radiation and is preferably likewise connected to one or the previously mentioned connection plugs, is provided in the supplementary module. This means that the modulated radiation emitted by the radiation source is likewise led into the fiber-optic of the endoscope via the connection plug, so that the modulated radiation exits from the endoscope at the distal side, preferably together with the light for illumination.
Furthermore, the supplementary module comprises a sensor, wherein the sensor receives modulated radiation or modulated light via the endoscope/supplementary module interface, said light having been reflected by the object to be examined with the endoscope. This means that the modulated radiation is led through the fiber-optic to the distal end of the endoscope, exits there and is incident on the object. The radiation is reflected by the object and enters into the distal-side end of the endoscope optics, which leads the radiation towards the proximal end and the interface to the supplementary module. The sensor is designed for detecting the radiation reflected by the object, at one or more picture points, in the manner described above by way of the endoscope, and preferably comprises several sensor elements for this.
The sensor is connected to an evaluation device which obtains readings of the detected radiation from the sensor. The evaluation device, on the basis of these readings and in particular with the knowledge of the modulation of the radiation source, determines phase differences at the one or the several picture points or pixels, respectively, and, on the basis of these, computes the distance of the object to the sensor device. The evaluation of the radiation detected by the sensor is effected in the manner described above by way of the endoscope system. The evaluation device may be integrated into the supplementary module, but is however preferably integrated into a control device which is connected to the supplementary module which may be attached directly onto the endoscope.
It is also alternatively possible to integrate the sensory devices for three-dimensional detection of objects directly into an endoscope or into endoscope optics, which may then be inserted into an endoscope shank, in particular of a conventional endoscope. It is possible also in this manner, to retrofit present endoscopes by way of merely exchanging the endoscope optics. With this embodiment, it is essentially those elements which are integrated into the endoscope optics which were previously described by way of the supplementary module. A radiation source is further provided, which emits a modulated radiation, in particular modulated light and may be connected to the endoscope such that an object to be examined by way of the endoscope may be illuminated with the radiation. Preferably, a connection device or a connection plug is provided for this, which may be connected to the fiber-optic cable connection of an endoscope, wherein the modulated radiation as well as the light for illuminating the object may be coupled into the fiber-optic of the endoscope via the connection plug in the manner described above.
Furthermore, a sensor for detecting the radiation reflected by the object, at one or more picture points, is integrated into the endoscope optics, wherein the detection of the radiation may be effected in the manner described above. The sensor, with regard to previously mentioned design, may preferably be arranged in the endoscope at the proximal side. An arrangement at the distal end, however, is also conceivable. Given an arrangement at the proximal end of an optical system, one may also provide a separation device or switch-over device, in order to be able to separate the signal for the three-dimensional measurement, from the usual picture signal. Moreover, the already described evaluation device is provided for the endoscope optics and this device obtains readings of the detected radiation from the sensor, and determines phase differences on the basis of these readings and in particular the information on the modulation of the radiation source at the one or more picture points, and computes the distance of the object to the sensor on the basis of these phase differences. This evaluation device may either be integrated directly into the endoscope optics or it is integrated into a control device, which is connected to the endoscope optics and the sensor device arranged on it, via a lead.
It is to be understood that the preferred features described above by way of the endoscope system may also be realized in the context of the described endoscope optics and the described supplementary module.
The foregoing summary, as well as the following detailed description of the invention, will be better understood when read in conjunction with the appended drawings. For the purpose of illustrating the invention, there are shown in the drawings embodiments which are presently preferred. It should be understood, however, that the invention is not limited to the precise arrangements and instrumentalities shown. In the drawings:
With regard to the endoscope system according to the invention or the additional module according to the invention or the endoscope optics according to the invention, one employs a measurement principle which is based on a TOF (time of flight) camera for the three-dimensional measurement of the object to be examined. The manner of functioning of this camera system is explained by way of
The pulsed light 10 is incident onto the object 2 and is sent back by this as reflected light 12 to the camera 4, and there is detected by the sensor 8 which comprises a multitude of sensor elements, of which each is allocated to an individual picture point or to a certain group of picture points, so that preferably essentially the whole object 2 is detected by the individual picture points.
The surface regions of the object 2 which face the camera 4 are distanced differently far from the camera 4 on account of the three-dimensional structure of the object 2. This leads to different transfer times of the light from the illumination device 6 back to the sensor 8. Longer transfer times of the light result for the picture points with which the object 2 or its surface is distanced further to the camera 4. The transfer time differences on account of the modulation of the emitted light 6 lead to phase differences which are detected and evaluated, in order to determine the transfer time and thus the distance of the picture points. In turn, one may then derive the three-dimensional structure of the object 2 from this. The manner of functioning of such a TOF-camera is known, for example, from DE 44 40 613 C1 or EP 1 622 200 A1.
According to the invention, this measurement principle which is applied by the previously mentioned TOF-cameras, is combined with an endoscope 2 which is represented schematically in
The endoscope 14 in the vicinity of its proximal end, comprises a fiber-optic cable connection 30 in the usual manner, which is connected to the proximal end of a fiber-optic in the inside of the endoscope 14, which ends at the distal end. The fiber-optic cable connection 30 according to the invention, on the one hand is connected to a conventional illumination device 32 which constantly emits light for illuminating the object 2, in order to produce a picture on the sensor 24. Furthermore, a modulated radiation source or light source 34 is provided, which emits modulated or pulsed light, which is likewise fed into the fiber-optic cable connection 30. Preferably, the light emitted by the illumination device 32 as well as the light source 34 is led together in a connection component with the help of known optical elements, so that the connection component then has only one exit, which is connected to the fiber-optic cable connection 30.
The radiation source 34 as well as the sensor device 20, are connected to a control device 35. The control device 35 comprises a clock generator, which sets the clock cycle for the pulse frequency and pulse lengths of the light emitted by the radiation source 34. Furthermore, the control device 35 detects the picture signals detected by the sensor 26 and evaluates these as an evaluation unit. The clock cycle of the light emitted by the radiation source 34 is known to control device 35, so that, on detecting the reflected light, this may then determine the transfer times for individual picture points on account of the phase differences, and in this manner compute the three-dimensional structure of the object 2. This may then likewise be displayed on the monitor 28. One may alternatively also provide a separate monitor. The picture processing for the picture recorded by the sensor 24 may likewise be integrated into the control device 35. Here, a separate picture processing system may likewise be provided as an alternative.
Two sensors 24 and 26 are provided in the example shown in
As is shown in
Sensors 24 and 26 and, as the case may be, a picture splitter 22 as well as necessary electronic components 46 for the control or regulation of the previously mentioned components are grouped together in the sensor device 20.
The modules of the sensor device 20, the illumination module 40 as well as the control device 35, may be provided as accessory components for use with a conventional endoscope 14. Thus a convention endoscope 14 may be upgraded into an endoscope, which permits the three-dimensional detection of structures.
The possible construction and the possible function of a picture splitter 22 is explained by way of the
A geometric beam splitting as is shown by way of
It will be appreciated by those skilled in the art that changes could be made to the embodiments described above without departing from the broad inventive concept thereof. It is understood, therefore, that this invention is not limited to the particular embodiments disclosed, but it is intended to cover modifications within the spirit and scope of the present invention as defined by the appended claims.
Number | Date | Country | Kind |
---|---|---|---|
07002065 | Jan 2007 | EP | regional |
Number | Name | Date | Kind |
---|---|---|---|
3299274 | Hoelter | Jan 1967 | A |
4415952 | Hattori et al. | Nov 1983 | A |
5534992 | Takeshima et al. | Jul 1996 | A |
5693003 | Wolfelschneider et al. | Dec 1997 | A |
5982555 | Melville et al. | Nov 1999 | A |
5990978 | Kim et al. | Nov 1999 | A |
6294775 | Seibel et al. | Sep 2001 | B1 |
20020040971 | Ono | Apr 2002 | A1 |
20020139920 | Seibel et al. | Oct 2002 | A1 |
20030223053 | Liu et al. | Dec 2003 | A1 |
20060034003 | Zalevsky | Feb 2006 | A1 |
20070109505 | Kubara et al. | May 2007 | A1 |
20070182949 | Niclass | Aug 2007 | A1 |
20080074552 | Jung et al. | Mar 2008 | A1 |
20110193987 | Wong et al. | Aug 2011 | A1 |
Number | Date | Country |
---|---|---|
44 40 613 | Jul 1996 | DE |
1 152 261 | Nov 2001 | EP |
1 452 125 | Oct 2004 | EP |
1 622 200 | Jan 2008 | EP |
Entry |
---|
Robert Lange et al., “Solid State Time-of-Flight Range Camera”, IEEE Journal of Quantum Electronics, vol. 37, No. 3, pp. 390-397, (2001). |
Number | Date | Country | |
---|---|---|---|
20080188716 A1 | Aug 2008 | US |