Patent Application
20140179997
The present application claims priority to German patent application serial no. DE 10 2012 025 100.9 filed Dec. 20, 2012, incorporated by reference herein in its entirety.
The present invention relates to a multi-camera system consisting of at least one endoscope camera and at least one trocar camera for use in minimally invasive procedures, as well as an appropriate surgical robot, especially for use in minimally invasive surgery, such as laparoscopy.
Minimally invasive surgical procedures, such as laparoscopic surgery performed with the use of surgical instruments, such as gripper tongs, cutting tools and sewing tools that are introduced via one or more trocars into the body of a patient. Usually two to four, and in most cases, three, surgical instruments are used. In addition to these surgical instruments, it is required that a display unit is present that allows the surgeon to observe the surgical field. Such a display unit is regularly a camera or an endoscope, which is also inserted through a trocar into the body of the patient. Usually, the visualization is made possible by an endoscope, which displays images of the surgical field in 2D or 3D on an external monitor. In the prior art, there exist numerous endoscopes, in which a display unit, such as a camera, is integrated in its distal end. Generally, endoscopes can have a camera at the distal as well as at its proximal end. The images obtained with the endoscope are displayed on one or more external monitors using an image transfer system and an image processing unit. Numerous endoscopes are described in the prior art.
The disadvantages of the camera systems or endoscopes described in prior art is that the endoscope is provided only for the visualization of the surgical field, but this endoscope cannot display at the same time the position and orientation of the surgical instruments inserted into the abdominal cavity, due to the varying positions the surgical instruments and the position of endoscopes in wider vicinity of the operation procedure and the field of view, whereby only the immediate area of the operation procedure is displayed. When a surgical instrument is removed from the surgical field of view, it is no longer detected by the endoscope and is no longer under the visual control of the surgeon or his assistant.
In one aspect, the present invention provides surgical robotic systems having at least two robot arms to each of which is arranged at least one endoscope,
wherein the first endoscope on the first robot arm comprises a main support means, which extends substantially over the entire endoscope length from outside into the interior of the body, and which comprises at the distal end at least one lighting unit and two image-taking devices, wherein the image-taking devices are each pivotally mounted to the outside essentially in the same plane of the main support means, and comprise a trocar, which accomplishes the access of the first endoscope inside the body, and
wherein the second endoscope on the second robot arm comprises a main support means, which extends essentially over the entire length of the endoscope from the outside into the interior of the body, a trocar, which accomplishes the access of the second endoscope inside the body, and an auxiliary support means, which is provided on the trocar and/or the main support means, wherein the additional support means comprises at its distal end an auxiliary image-taking device, which is arranged pivotally from the additional support means to the outside, and wherein the additional image-taking device comprises an auxiliary lighting unit and at least one additional image sensor having a monitoring region, which includes the two monitoring areas of the image-taking means of the first endoscope,
wherein an image processing unit, which is connected both to the two imaging devices and the additional image-taking device, and a visualization unit is provided, which displays the 2D image data, and/or the 3D image data of the image-taking means and/or the additional image-taking means.
In one embodiment, the additional image sensor has a wide-angle lens, which in the pivoted state is arranged close to the distal end of the trocar. In another embodiment, the two imaging devices are mounted at the distal end of the main support means pivotally about a pivot axis, wherein the pivot axes are parallel to each other in one plane. In a further embodiment, the additional support means is abutting between the trocar (1a) and the main support means, in particular directly onto the main support means, wherein in particular both the main support means and the additional support means are formed cylindrical. In another embodiment, the image-taking devices are arranged by means of joints so that in each case they can be tilted both about the pivot axis and a further axis of rotation orthogonally to the longitudinal extension of the main support means, wherein the rotation about the pivot axes and the rotational axes are independent being decoupled from each other. In a still further embodiment, (i) at least one third endoscope is provided on a third robot arm, which comprises a main support means, which extends substantially over the entire endoscope length from the outside into the body, (ii) a trocar, which accomplishes the access of the third endoscope inside the body, and (iii) an additional support means, which is arranged on the trocar and/or the main support means, wherein the additional support means comprises at its distal end an additional image-taking means, which is arranged pivotably from the additional support means to the outside, and wherein the additional image-taking device comprises an additional lighting unit and at least one additional image sensor, which has a monitoring area that includes the two monitoring areas of the image-taking means of the first endoscope. In yet another embodiment, the additional imaging means of the third endoscope is connected to the image-processing unit, and the display unit displays the 2D image data and/or the 3D image data of the image-taking means and/or the additional image-taking means and/or the additional image-taking means.
Purely by way of example, the present invention is now illustrated by the accompanying figures.
The technical task of the invention therefore is to provide an improved visualization system for minimally invasive surgical procedures, such as laparoscopic procedures, which provides the surgeon with the additional information about the location and orientation of instruments introduced into the patient, for example through the abdomen. This task is solved by the present invention according to claim 1 by a surgical robot system with at least two corresponding endoscopes.
The present invention provides a decoupled multi-camera system consisting of an endoscope and at least one further trocar camera for use in minimally invasive surgical procedures, such as laparoscopy.
A first subject of the present invention relates to a trocar camera for minimally invasive surgery, in particular for use within a surgical robot system with at least two robot arms, on each of which are arranged at least one endoscope for minimally invasive surgery,
wherein the first endoscope comprises in the first robot arm a main support means, which extends essentially over the entire endoscope length from the outside into the interior of the body, and which comprises at the distal end at least one lighting unit and two image-taking devices, wherein the image-taking devices are each pivotally mounted essentially in the same plane as the main support means and comprises a trocar, which accomplished the access of the first endoscope inside the body, and
wherein the second endoscope on the second robot arm comprises a main support means extending essentially over the entire endoscope length from outside into the interior of the body,
a trocar, which accomplished the access of the second endoscope inside the body, and
an additional support device, which is provided on the trocar and/or the main support means, wherein the additional support means comprises at its distal end an additional image-taking means, which is pivotally mounted on the additional support means to the outside and wherein the additional image-taking means comprises an auxiliary lighting unit and at least one auxiliary image sensor, which comprises a surveillance area that includes the two surveillance areas of the of the image-taking devices of the first endoscope,
wherein an image-processing unit, which is coupled both to the two image-taking devices and the additional of image-taking means (8, 9, 10, 11), and a visualization unit, which displays the 2D image data and/or 3D image data of the image-taking devices and/or the additional image-taking means.
The present invention has the advantage that by the provision and the simultaneous use of at least two imaging systems, at least one at least 2D vision camera at an endoscope and a 3D detail camera at another endoscope, wherein for the introduction of the 2D view camera a combination trocar is used together with a surgical instrument and are introduced into the body of a patient, it is possible to generate both an at least 2D view image with a broad field of view (wide angle of typically >90°) and a 3D detail image with a usual field of view of up to 70°. This makes it possible to monitor the direct operation area and its wider area for the entire duration of a minimally invasive surgical, such as laparoscopic, surgery. In this way, all surgical instruments can be imaged simultaneously, even if—due to their varying positions and the position of the endoscope and the field of view—they are located outside the operating field of view of the endoscope, because the additional image-taking device can detect also instruments, which are located outside of the surgical field of view of the endoscope. This may be the case, for example, when a surgical instrument is temporarily “parked”, because it is not needed. This “parking” is in most cases out of the direct operation procedure and outside of the operation field of view so that it is not in the way during the procedure. According to the invention, such “parked” surgical elements are captured by the novel 2D surveillance camera and are thus continuously under visual control of the surgeon or his assistant.
Furthermore, the additional illumination unit on the additional support means achieves improved illumination in particular for the 3-D images so that the images of the 3D-detail camera can be represented qualitatively improved.
According to a preferred embodiment of the invention, the additional image sensor has a wide-angle lens, which in the pivoted state is arranged close to the distal end of the trocar.
A third subject of the present invention relates to a surgical robot system with at least two robot arms, on which a surgical instrument and an endoscope for minimally invasive surgery can be disposed, wherein in addition, on the trocar for the surgical instrument an additional support means is arranged with the additional image-taking device, a trocar, which accomplishes the access of the endoscope inside the body, and wherein the additional support means comprises at its distal end an additional image-taking device, which is mounted pivotally from the auxiliary support means to the outside, and wherein the additional image-taking means comprises an auxiliary lighting unit and at least one additional image sensor, which has a surveillance area that includes all surgical instruments and/or endoscopes in their positions and orientation that were introduced, for example, through the abdomen,
wherein an image-processing unit, which is coupled both to the endoscope and the additional image-taking device, and a display unit is provided, which represents 2D image data, and/or 3D image data of the endoscope and/or the additional of image-taking means.
A control unit knows the current position and orientation of the robot arms and the attached instruments or endoscopes, and this information is sent to an image-processing unit, which is coupled both to the endoscope and the additional image-taking device, and a display unit is provided that displays the 2D image data and/ or 3D image data of the endoscope and/or the additional image-taking means and, in addition, calculate from the position and orientation of the robot arms and the attached instruments or endoscope the movement trajectories and displays them as an overlay display along with the 2D image data and/or 3D image data.
The inventive surgical robot system has the particular advantage that the image data can be displayed to the surgeon as required, that is either as a 2D image data or as 3D image data, which means that the image data of the surveillance camera can also be coupled with the image data of the 3D-detail camera using the image-processing unit so as to provide the surgeon with a much improved overview by a single sequence of images on the display unit. The surgical robot system can be used, for example, in conducting minimally invasive surgery.
It is particularly advantageous when the additional image sensor has a wide-angle lens, which in the swung-out state is arranged close to the distal end of the trocar.
It is particularly advantageous if the two image-taking devices are each arranged at the distal end of the main support means pivotally mounted about a rotational axis, wherein the pivot axes are parallel to each other in one plane, whereby the construction cost is minimized.
A further structural simplification can be seen in the fact that the additional support means is provided between the trocar and the main support means, in particular directly abutting the main support means, wherein in particular both the main support means and the additional support means are cylindrical.
Furthermore, it is advantageous if the image-taking devices are tiltably arranged, by means of hinges, in each case both to the pivot axis as well as a further rotation axis orthogonal to the longitudinal extension of the support means, wherein the rotational movements about the pivot axes and the axes of rotation are independently decoupled from each other.
A further preferred embodiment of the invention is characterized in that at least a third endoscope is provided on a third robotic arm, which comprises a main support means, which extends essentially over the entire endoscope length from outside into the interior of the body, a trocar, which allows access of a third endoscopes inside the body, and an additional support means, which is provided on the trocar and/or the main support means, wherein the additional support means comprises at its distal end an additional image-taking means, which is arranged pivotally from the additional support means to the outside, and wherein the additional image-taking means comprises an additional illumination unit and at least one additional image sensor, which captures a monitoring area that includes the two control areas of the image-taking devices of the first endoscope.
Further, it is advantageous if the additional image-taking device of the third Endoscopes coupled to the image processing unit, and the display unit is the 2D image data, and/or 3D image data of the image-taking devices and/or the addition of image-taking device and/or the addition of image-taking means.
The entire disclosure of the present invention therefore relates equally to the combination consisting of a 3D-detail camera and at least one further 2D-vision camera, which is introduced into the interior of the body preferably through a further trocar used for a surgical instrument and is positioned in such a way that all further surgical instruments or endoscope cameras introduced into the body through other trocars are covered by the 2D surveillance camera optically, as well as on the combination of a 3D detail camera with at least two 2D vision cameras which preferably has two further, for surgical instruments used Trocar e, are introduced into the interior of the body and are positioned in such a way that all imported via further trocar e in the body surgical Instruments and Endoscope cameras are optically covered by the 2D surveillance camera.
In minimally invasive surgical procedures, such as laparoscopic surgery, access into the body of a patient (usually through the abdomen or the chest cavity) is created through a trocar. Using such a trocar, a surgical instrument or a camera or an endoscope can be introduced into the body. As mentioned, according to the invention a surgical instrument and a trocar camera are simultaneously introduced through a trocar. Since, as a rule, 3 to 5 surgical instruments and at least one camera are required for the surgical intervention, 3 to 5 trocars are needed.
The received image data is supplied via the data link 29 to a processing unit 31, which processes the image data for display and supplies it via a further data path 32 to a visualization unit 33. The visualization unit 33 can display both 2D and 3D image data, for example separately, but also combined into a single image or a single image sequence.
At the end of the rotationally symmetrical main support 4b are located two camera modules and two image-taking devices 20a, 21a, 22a, 20b, 21b, 22b that each consist in particular of two optical imaging systems 22a and 22b that are mounted on two camera holders 20a and 20b. The camera holders 20a and 20b are so connected to the main support 4b by means of the joints 25a and 25b, which form the pivot axes, that after the introduction into the body, they can be folded out by 90° to the rotation axis of the main support 4b in the pivoting directions 26a and 26b. In order to illuminate the field of view, at the end of the main support 4b, to which are also attached the folding camera holders 20a and 20b, there is mounted a lighting unit consisting of the light source 23 and an imaging optical system 24. The camera holders 20a and 20b continue to wear the imagers consisting of image sensors 21a and 21b and imaging optics 22a and 22b. Together, these two image-taking devices 20a, 21a, 22a, 20b, 21b, 22b form the 3D-detail camera.
The lighting unit consisting of a light source 23 and imaging optics 24 can be preferably formed as a direct LED light source in such a way that the emission angle of the LED in conjunction with a suitable imaging lens 24 is so selected that the field of view displayed by two image sensors 21a and 21b and the associated imaging optics 22a and 22b is completely illuminated.
The received image data is supplied via the data link 30 to a processing unit 31, which processes the image data for display and supplies it via a further data path 32 to a visualization unit 33. The visualization unit 33 can display both 2D and 3D image data, for example separately, but also combined into a single image or a single image sequence.
The received image data is supplied through the data lines 29, 30 to a processing unit 31, which processes the image data for display, and supplies it via a further data path 32 to a visualization unit 33. The visualization unit 33 can display both 2D and 3D image data, for example separately, but also combined into a single image or a single image sequence.
The control and display unit 34 is connected by a suitable data link 52 to the processing unit 31. Using the control and display unit 34, the surgeon can send control commands for selection, processing and display of the image data to the processing unit 31. The processing unit 31 is connected by a suitable data link 32 to the display unit 33. The display unit 33 can display detail images/image sequences supplied by the 2D surveillance camera and the 3D camera, as well as additional information generated in the processing unit 31, such as trajectories of the surgical instruments, either as separate images or image sequence or as image information calculated by the 2D surveillance camera and/or the 3D-detail camera.
The received image data is supplied through the data line 29a to a processing unit 31, which processes the image data for display and supplies it via a further data path 32 to a visualization unit 33. The visualization unit 33 can display both 2D and 3D image data, for example separately, but also combined into a single image or a single image sequence.
A trocar 1d provides a passage through the body tissue 2 and, therefore, access to the interior of the body of a patient. An additional support 3d for a second 2D vision camera is introduced into the body through the trocar 1d. The additional support 3d is designed so that it enables a tubular passage for a further rotationally symmetrical rod-shaped main support 4d for a surgical instrument. To the auxiliary support 3d is attached a camera holder 5d by means of a hinge 6d so that after passage through the trocar 1d, it can be folded out in a pivoting movement essentially 90° to the rotation axis. The camera holder 5d carries an additional image sensor consisting of image sensor 9d and wide-angle imaging optics 8d with the opening angle 8d. In order to illuminate the field of view, the camera holder 5d is further provided with an additional lighting unit 11d consisting of a light source 11d and a corresponding wide-angle imaging optical system 10d with the opening angle 19d. This wide-angle imaging optical system 10d is designed such that, except for the parallax offset between the wide-angle imaging lens 8d and the wide-angle imaging optical system 10d, the whole field of view of the image sensor 9d and the associated wide angle imaging optics 8d is illuminated. The camera holder 5d with the additional image sensor and the additional lighting unit together form the second 2D view camera for generating a second 2D overview image. Preferably, the image sensor 9d is designed as a CCD or CMOS sensor with a resolution of 1920×1080 pixels or higher. With a suitable positioning of the second 2D surveillance camera at an outer trocar 1d, all further surgical instruments or endoscopes introduced into the body through the trocars 1a, 1b, 1c are in the field of view of the second 2D surveillance camera and can be visually captured by it and imaged onto the image sensor 9d.
The received image data is supplied by the data link 31 to a processing unit 29d, which processes the image data for display, and supplies it via a further data path 32 to a visualization unit 33. The visualization unit 33 can display both 2D and 3D image data, for example separately, but also combined into a single image or a single image sequence.
At the end of the rotationally symmetrical main support 4b are located two camera modules or two image-taking devices 20a, 21a, 22a, 20b, 21b, 22b that each consist in particular of two optical imaging systems 22a and 22b that are mounted on two camera holders 20a and 20b. The camera holders 20a and 20b are connected—by means of the joints 25a and 25b, which form the pivot axes—to the main support 4b that, after the introduction into the body, it can be folded out by 90° to the rotation axis of the main support in the pivoting directions 26a and 26b. In order to illuminate the field of view, at the end of the main support 4b, to which are also attached the folding camera holders 20a and 20b, there is provided a lighting unit consisting of the light source 23 and an imaging optical system 24. The camera holders 20a and 20b further carry imager consisting of image sensors 21a and 21b and imaging optics 22a and 22b. Together, these two image-taking devices 20a, 21a, 22a, 20b, 21b, 22b form the 3D-detail camera.
The lighting unit consisting of a light source 23 and the imaging optics 24 can preferably be realized as a direct LED light source in such a way that the emission angle of the LED in conjunction with a suitable imaging lens 24 can be selected so that the field of view formed by the two image sensors and 21a and 21b and the associated imaging optics 22a and 22b is completely illuminated.
The received image data is supplied by the data link 30 to a processing unit 31, which processes the image data for display and supplies it via a further data path 32 to a visualization unit 33. The visualization unit 33 can display both 2D and 3D image data, for example separately, but also combined into a single image or a single image sequence.
The received image data is fed through the data lines 29a, 29b, 30 to a processing unit 31, which processes the image data for display and supplies it via a further data path 32 to a visualization unit 33. The visualization unit 33 can display both 2D and 3D image data, for example separately, but also combined into a single image or a single image sequence.
The control and display unit 34 is connected by a suitable data link 52 to the processing unit 31. Using the control and display unit 34, the surgeon can send control commands for selection, processing and display of the image data to the processing unit 31. The processing unit 31 is connected by a suitable data link 32 to the display unit 33. The display unit 33 can be supplied by the 2D surveillance camera and the 3D-detail camera with images or image sequences as well as additional information generated in the processing unit 31, such as the trajectories of the surgical instruments that are represent either as separate images/image sequence or as images or image sequences calculated by the image information of the 2D surveillance camera and/or the 3D-camera detail.
Thus, the present invention describes a surgical robot system, in which the trajectories of the surgical instruments and the lighting devices are displayed on a display of the display unit so that the surgeon is informed not only of the current position of the individual elements of the instruments but also gets displayed, in which direction are other instruments and lighting equipment. Thus, the present invention makes it possible for the surgeon to always coordinate all instruments and not to have to bring them to the field of view of the 3D camera in a “blind flight”.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2012 025 100.9 | Dec 2012 | DE | national |
