Patent Grant
11986197
Embodiments relate to an autonomous endoscopic system and control method therefor.
An endoscope is a medical instrument used to directly investigate internal organs or the inside of body cavities. The endoscope is designed to be inserted into the body to observe an organ with a lesion that cannot be directly seen without performing surgery or autopsy. There are many types of endoscopes such as, for example, a bronchoscope, an esophagoscope, a gastroscope, a duodenoscope, a proctoscope, a cystoscope, a laparoscope, and a ureteroscope.
For example, endoscopic surgery using a ureteroscope is known as the most frequently used, reliable method to remove kidney stones. In detail, the method is performed in the manner of inserting the ureteroscope through the ureter to reach a kidney using radiodiagnostic equipment such as C-arm, fragmenting stones with a laser, and extracting the stones using a basket. Here, the laser and the basket are inserted through a channel inside the ureteroscope.
Meanwhile, the radiodiagnostic equipment should be continuously used to check the position of the endoscope in the body. Thus, doctors and patients are at risk of radiation exposure. In particular, surgery is highly difficult since a very slender endoscope like a ureteroscope has a limited degree of freedom (bending 1 DOF), and there may be communication issues between two operators who need to perform precise works together. In addition, the surgery precision may decrease when the arms of an operator are fatigued over time as the operator should hold the endoscope for a long time. As described above, the endoscopic surgery requires precision, which results in the high fatigue of the operator. In addition, since it is impossible to accurately measure the size of stone using the endoscope alone, there may occur a medical accident such as hurting the ureter of a patient in the process of withdrawing a stone that is not fragmented sufficiently. If the ureter is damaged, surgery is immediately performed through an incision, and after that, there may be fatal aftereffects.
In consideration of such issues, there is a need for endoscopic equipment that may reduce the risk of radiation exposure of doctors and patients, reduce fatigue, and prevent medical accidents.
The above description has been possessed or acquired by the inventor(s) in the course of conceiving the present invention and is not necessarily an art publicly known before the present application is filed.
An aspect provides an autonomous endoscopic system and control method therefor.
According to an aspect, there is provided an autonomous endoscopic system capable of controlling movement of an endoscope inserted into a protective sheath installed in the body of a patient, the autonomous endoscopic system including an endoscope operating device capable of adjusting a relative position of the endoscope with respect to the protective sheath, a rolling angle of the endoscope, and a bending angle of a bending portion which is located at the end of the endoscope and is bendable, and a control unit for controlling the endoscope operating device, wherein the control unit may control the endoscope operating device on the basis of a driving record of the endoscope.
The autonomous endoscopic system may further include a sensor for sensing information about the relative position of the endoscope with respect to the protective sheath, wherein the driving record may include information about a relative movement of the endoscope with respect to the protective sheath.
The sensor may include a first magnetic body and a second magnetic body installed on the protective sheath and the endoscope, respectively.
A basket for gripping a stone present inside the body of the patient may be inserted into the endoscope, and the control unit may control the endoscope operating device such that the endoscope automatically returns to the site where the endoscope was located at a point in time the basket completes a gripping operation.
A basket for gripping a stone present inside the body of the patient may be inserted into the endoscope, and when the basket completes a gripping operation, the control unit may control the endoscope operating device (a) to withdraw the endoscope from the protective sheath, (b) to release the gripped stone by opening the basket, and (c) to cause the endoscope to re-enter, such that the endoscope automatically returns to the site where the endoscope was located at a point in time the basket completes the gripping operation.
The driving record may include an operation profile that shows an operation amount of the endoscope operating device over time from a first point in time to a second point in time.
The first point in time may be a point in time the end of the endoscope is located at a particular site with respect to the protective sheath.
The second point in time may be a point in time at which a surgical instrument inserted into the endoscope performs a particular work.
At least one of the first point in time and the second point in time may be an optional point in time to be set by an operator.
The operation amount may include a translation amount of the endoscope, a rolling angle variation of the endoscope, and a bending angle variation of the endoscope.
The control unit may (a) determine whether there is, between the first point in time and the second point in time, a forward-backward movement interval during which the endoscope passes a particular position and then returns again, (b) generate, if there is a forward-backward movement interval, a shortened profile by removing an operation amount over time during the forward-backward movement interval from the operation profile, and (c) control the endoscope operating device according to the shortened profile.
The control unit may (a) determine whether there is, between the first point in time and the second point in time, a forward-backward movement interval during which the endoscope passes a particular position and then returns again, (b) generate, if there is a forward-backward movement interval, a corrected profile that includes a rolling angle variation and a bending angle variation during the forward-backward movement interval, that does not include a translation amount during the forward-backward movement interval, and that is performed for a time shorter than the forward-backward movement interval, (c) generate a shortened profile by replacing an operation amount over time during the forward-backward movement interval in the operation profile with the corrected profile, and (d) control the endoscope operating device according to the shortened profile.
The control unit may initialize a pose of the endoscope such that the endoscope has a particular rolling angle and a particular bending angle when the end of the endoscope is located at a particular site with respect to the protective sheath.
The autonomous endoscopic system may further include a display for outputting an image of the inside of the body of the patient to an operator, wherein the control unit may display the image by overlaying, on the image, an expected position and pose of the endoscope at a point in time after a set time elapses from a current time.
The autonomous endoscopic system may further include a clutch operable by an operator and capable of allowing or stopping a continuous drive of the endoscope operating device.
The autonomous endoscopic system may further include a master device located at a site spaced apart from the endoscope operating device and operated by an operator to remotely operate the endoscope operating device.
According to embodiments, an endoscope may autonomously drive to a particular position based on kinematic driving records, and thus it is possible to remarkably reduce the surgery fatigue of an operator.
According to embodiments, particularly in the case of kidney stone surgery, it is possible to automatically perform repeated works such as repeatedly inserting and withdrawing an endoscope to withdraw a number of stone fragments at a particular site one by one.
According to embodiments, particularly in the case of kidney stone surgery, it is possible to quickly perform work such as repeatedly accessing a particular position, which is very difficult due to the complex internal structure, based on previous driving records. Further, it is possible to access the particular position along the shortest path based on the previous driving records, and thus the operation time may be considerably reduced. Consequently, the time for general anesthesia of a patient may be reduced, which may improve the stability of surgery, in particular, for elderly patients. According to the simulation results, the operation time of 2 hours on average may be reduced to 1 hour or less.
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Regarding the reference numerals assigned to the components in the drawings, it should be noted that the same components will be designated by the same reference numerals, wherever possible, even though they are shown in different drawings. Also, in the description of the embodiments, detailed description of well-known related structures or functions will be omitted when it is deemed that such description will cause ambiguous interpretation of the present disclosure.
Also, in the description of the components, terms such as first, second, A, B, (a), (b) or the like may be used herein when describing components of the present disclosure. These terms are used only for the purpose of discriminating one constituent element from another constituent element, and the nature, the sequences, or the orders of the constituent elements are not limited by the terms. When one constituent element is described as being “connected”, “coupled”, or “attached” to another constituent element, it should be understood that one constituent element can be connected or attached directly to another constituent element, and an intervening constituent element can also be “connected”, “coupled”, or “attached” to the constituent elements.
The same name may be used to describe an element included in the embodiments described above and an element having a common function. Unless otherwise mentioned, the descriptions on the embodiments may be applicable to the following embodiments and thus, duplicated descriptions will be omitted for conciseness.
Referring to
The endoscope 910 may include a control handle 911 to be gripped by an operator with a hand, an instrument hole 912 configured to guide the surgical instrument 920, an insertion tube 913 connected to the control handle 911 and to be inserted into the body, a bending portion 915 located at the end of the insertion tube 913 and configured to perform a bendable motion, a knob 914 provided rotatably in the control handle 911 and configured to adjust an angle of the bending portion 915 according to manipulation, and a magnetic body 916 provided in the insertion tube 913. The function of the magnetic body 916 will be described later.
The surgical instrument 920 may include an operation part 921 to be gripped with a hand and operated by the operator, a surgical instrument cable 922 connected to the operation part 921 and to be inserted into the insertion tube 913 of the endoscope 910, and an action part 923 provided at the end of the surgical instrument cable 922 and to be operated by the operation part 921. The action part 923 may include, for example, a basket capable of gripping a stone present inside the body of a patient. Hereinafter, a case where the action part 923 is a basket will be described as an example. However, differently, the action part 923 may be another means known to those skilled in the art, such as a laser lithotripter that fragments stones.
For example, the operation part 921 may include a first operation unit 921b and a second operation unit 921b, wherein the operation part 921 may operate the action part of the surgical instrument 920 by moving the second operation unit 921b, which moves relatively with respect to the first operation unit 921b, toward the first operation unit 921b. According to the operation of the operation part 921, the action part 923 may perform a particular work (for example, operating the basket or operating the laser lithotripter).
Referring to
The protective sheath 11 may be placed in the body of the patient to prevent damage to the body of the patient by friction with the endoscope 910. The protective sheath 11 includes a protective hole provided in a length direction thereof to guide the endoscope 910.
The endoscope operating device 12 may provide a driving force to drive, roll or bend the endoscope 910, or provide a driving force for the surgical instrument 920 to perform a particular work. For example, the endoscope operating device 12 may have a structure that is attachable to and detachable from a commercial endoscope and/or a commercial surgical instrument. The endoscope operating device 12 may have a structure that is physically fastenable to each operation part of the commercial endoscope and/or the commercial surgical instrument, and may operate the commercial endoscope and/or the commercial surgical instrument based on a signal input through the master device 15 by driving the fastened part. Meanwhile, instead of using a commercial endoscope and/or a commercial surgical instrument, an endoscope and/or surgical instrument dedicated to the endoscope operating device 12 may be used, and the endoscope operating device 12 may be formed integrally with the endoscope and/or surgical instrument. The endoscope operating device 12 may include a translation operation part 121, a rolling operation part 122, a bending operation part 123, and a surgical instrument operation part 124.
The translation operation part 121 may adjust a relative position of the endoscope 910 with respect to the protective sheath 11. For example, the translation operation part 121 may include a driving source configured to move the entire endoscope 910 forward or backward by moving, along a linear guide rail, a block with the control handle 911 of the endoscope 910 mounted.
The rolling operation part 122 may manipulate a rolling angle of the endoscope 910. For example, the rolling operation part 122 may include a driving source configured to rotate, based on an axial direction of the endoscope 910, the block with the control handle 911 of the endoscope 910 mounted.
The bending operation part 123 may adjust a bending angle of the bending portion 915 of the endoscope 910. For example, the bending operation part 123 may include a driving source configured to pull a wire inserted into the bending portion 915.
The surgical instrument operation part 124 may include, on the block with the control handle 911 of the endoscope 910 mounted, a drive source configured to relatively move with respect to the block and drive another block with the surgical instrument 920 mounted.
Meanwhile, the above description is only an example of the endoscope operating device 12. As another example, unless otherwise mentioned, any slave device operable with a master-slave relationship may be used as the endoscope operating device 12, for example, as in KR 1882093, JP 2010-279688, and JP 2007-117394.
The control unit 13 may control the endoscope operating device 12. The control unit 13 may operate the endoscope 910 and the surgical instrument 920 by controlling the endoscope operating device 12 based on control signals input through the sensor 14, the master device 15, the input unit 16, and/or the clutch 17. The control unit 13 may enable the endoscope 910 to autonomously drive by controlling the endoscope operating device 12 based on a driving record of the endoscope 910, which will be described later with reference to
The sensor 14 may sense information about the relative position of the endoscope 910 with respect to the protective sheath 11. The control unit 13 may control the endoscope operating device 12 based on the information sensed by the sensor 14.
For example, the sensor 14 may include a first magnetic body 111 (see
As another example, the sensor 14 may be a displacement sensor connected between the protective sheath 11 and the endoscope 910 to sense a change in relative position.
As still another example, the sensor 14 may sense a translation amount, a rolling angle, a bending angle, and/or an operation of the endoscope 910 (for example, whether an operation of gripping a basket is performed), by sensing the operation amount of the endoscope operating device 12.
In the present application, the sensor 14 may include any means capable of sensing the information about the relative position of the endoscope 910 with respect to the protective sheath 11, in addition to the means exemplarily proposed above. For example, the radiodiagnostic device 18 may function as the sensor 14, which will be described later.
The master device 15 may be located at a site spaced apart from the endoscope operating device 12 and operated by an operator to remotely operate the endoscope operating device 12. The master device 15 shown in
The input unit 16 may receive an instruction from an operator and transmit the instruction to the control unit 13. For example, the input unit 16 may include a known user interface, such as a keyboard or a mouse. The operator may select an interval to store the driving record of the endoscope 910 through the input unit 16.
The clutch 17 may be operable by the operator, and receive the instruction from the operator and transmit the instruction to the control unit 13, thereby allowing or stopping a continuous drive of the endoscope operating device 12. For example, the clutch 17 may have a foot pedal structure which allows input of information through a foot, rather than using a hand of a user, as shown in
The radiodiagnostic device 18 may capture and image the inside of the body of the patient. The radiodiagnostic device 18 may provide the operator with the position of the endoscope 910. For example, a C-arm may be utilized as the radiodiagnostic device 18.
By the radiodiagnostic device 18, the position of the protective sheath 11 may be easily known. Further, the protective sheath 11 may maintain a fixed position inside the body of the patient irrespective of the movement of the endoscope 910, and the protective sheath 11 and the endoscope operating device 12 may remain fixed to each other through a fixing tool. By the structure described above, it is possible to collect the relative position between the protective sheath 11 and the endoscope 910 through image processing from an image obtained through the radiodiagnostic device 18, and drive the endoscope 910 based on the collected information. In other words, when the radiodiagnostic device 18 is used, it is possible to operate the endoscope 910 even without using the magnetic bodies 111 and 916 as described above. In detail, the control unit 13 may sense the relative position of the endoscope 910 with respect to the protective sheath 11 based on the image obtained using the radiodiagnostic device 18. In this regard, the radiodiagnostic device 18 may be construed as being included in the sensor 14.
The display 19 may provide the operator with the image of the inside of the body of the patient through a camera mounted on the radiodiagnostic device 18 and/or the endoscope 910. For example, the operator may select, on the display 19 by a screen touch or mouse click, the end of the protective sheath 11 and the position of a particular part (for example, a minor calyx) of the organs of the patient, and the control unit 13 may extract a relative distance between the two selected points and provide the operator with the extracted relative distance.
For example, in an autonomous driving mode, the control unit 13 may determine an expected position and pose of the endoscope 910 at a point in time after a set time elapses from a current time, and display the expected position and pose through the display 19 by overlaying a semitransparent image of the expected position and pose, on the image of the inside of the body of the patient. In this state, the operator may operate the clutch 17 to determine whether to continue allowing the autonomous driving of the endoscope 910 without an additional operation, or stop or end the autonomous driving as necessary and directly drive the endoscope 910 using the master device 15. By the configuration described above, when an error occurs due to aging and a degree of bending angle of the endoscope 910, the operator may correct the error by intervening in the adjustment, whereby the stability of surgery may improve significantly. Further, the control unit 13 may collect the information corrected by the operator intervening in the adjustment and use the collected information as big data of a deep learning algorithm to reduce driving errors of the endoscope 910.
The radiation shield wall W may be provided between an area where the operator is located and an area where the patient is located. In other words, the radiation shield wall W may be provided between the master device 15 and the endoscope operating device 12 to separate two areas, that is, the area where the operator is located and the area where the patient is located. By the radiation shield wall (W) described above, it is possible to reduce the risk of radiation exposure by the radiodiagnostic device 18 affecting the operator who performs surgery on a large number of patients.
Referring to
Further, once a kidney stone (ks) is found, a process of fragmenting the kidney stone and repeatedly withdrawing a number of fragments of the stone from the body of the patient using the basket 923 (see
In the state in which the stone is fragmented, the endoscope 910 needs to move tens of times along the same path to the same minor calyx (mc) to withdraw the stone fragments, and thus, the fatigue of the operator is continuously accumulated. In addition, if the operator places, in reality, the endoscope 910 in another minor calyx (mc), rather than placing the endoscope 910 at the previous work position in the same minor calyx (mc), the operation time may increase, or the surgery may not be performed perfectly.
Meanwhile, since the kidney (k) is a relatively solid organ compared with the other organs, the kidney (k) is generally maintained in the same position during the surgery. Consequently, except the basket job (b) of the procedure shown in
Referring to
First, in operation S11, the operator may insert the protective sheath 11 into the body of the patient. The protective sheath 11 may be inserted to pass through the urethra, the bladder, and the ureter of the patient such that the end thereof is located in the renal pelvis of the kidney.
In operation S12, the endoscope operating device 12 may translate the endoscope 910 along the inside of the protective sheath 11. Meanwhile, through information sensed by the sensor 14 during the translation movement of the endoscope 910, the control unit 13 may control the endoscope operating device 12 such that the end of the endoscope 910 is located at a particular site with respect to the protective sheath 11 and/or at a particular rolling angle. This process may be performed regardless of the driving record of the endoscope 910. Operation S12 may also be referred to as an initialization operation. For example, the control unit 13 may initialize the pose of the endoscope 910 such that the endoscope 910 has a particular rolling angle and a particular bending angle when the end of the endoscope 910 is located at a particular site with respect to the protective sheath 11. The particular site may be the renal pelvis from which it is easy for the end of the endoscope 910 to access most of the minor calyces.
In operation S13, the operator may scan for a stone by operating the endoscope 910 using the master device 15. If a stone is found, the operator may fragment the stone by operating the surgical instrument 920 using the master device 15, in operation S14. Thereafter, in operation S15, while the endoscope 910 is maintained at the same position and pose, the operator may replace the surgical instrument 920 such as a stone fragmenting tool (for example, a laser lithotripter) with a stone gripping tool (for example, a basket). In operation S16, the operator may perform the basket job of gripping the stone.
While the operator operates the master device 15, the driving record of the endoscope 910 from a first point in time to a second point in time may be stored, in operation S21. In operation S21, an operation amount of the endoscope 910 operated by the endoscope operating device 12 may be recorded over time. Such an operation amount over time may be referred to as an “operation profile”. Meanwhile, the “operation amount” may include a translation amount of the endoscope 910, a rolling angle variation of the endoscope 910, and a bending angle variation of the endoscope 910. Examples of the operation profile described above are shown in
For example, as shown in
As another example, as shown in
If a stone is gripped in operation S16, the operator may operate the master device 15 to move the endoscope 910 backward together with the stone, or automatically operate the endoscope operating device 12 by transmitting, to the control unit 13 through the input unit 16, information indicating that the stone is gripped, thereby withdrawing the endoscope 910 from the protective sheath 11 in operation S17 and removing the stone by releasing the stone from the surgical instrument 920 in operation S18.
In operation S19, the control unit 13 may receive, from the operator, confirmation regarding whether the stone removal has been completed at the previous work position at which the basket job has been performed.
If information indicating that there is a stone remaining at the previous work position is input in operation S19, the control unit 13 may automatically cause the endoscope 910 to autonomous drive based on the existing driving record of the endoscope 910 collected in operation S21, thereby inserting the endoscope 910 again such that the end of the endoscope 910 is located at the same site of the previous work position, in operation S22. As such, operations S16 to S19 and S22 may be repeatedly performed until all the stones are withdrawn from the position at which the basket has gripped the stones. Through operation S22, the operator does not need to memorize the path that the endoscope 910 has passed to reach the previous work position and operate the master device 15 along the path. Since the operator only needs to perform the basket job (operation S16), the fatigue of surgery may be significantly reduced. In addition, since there is no trial and error that may occur until the operator finds the previous work position, the operation time may be significantly reduced.
When information indicating that the stone removal has been completed at the previous work position is input in operation S19, the control unit 13 may receive, from the operator, confirmation regarding whether the stone removal has been completed at all work positions, in operation S20.
When information indicating that there is a work position at which the stone removal is yet to be performed is input in operation S20, operation S12 may be performed, and sequentially operation S13 of scanning for stones may be performed.
Such a phenomenon may occur, for example, when the endoscope 910 is stuck in the inner wall of an organ of a patient and receives resistance. Nevertheless, if the endoscope 910 autonomously drives to the previous work position through repetition of the same operation using the operation profile without correction, repeated shocks are applied to the organ of the patient, which may cause a medical accident. In addition, the trial and error time for the forward and backward movements unnecessarily increases the operation time. Thus, embodiments for resolving this issue will be described below.
Referring to
Here, the “forward-backward movement interval” refers to an interval, in the operation profile, during which the end of the endoscope 910 passes a particular position and returns again, and may be the interval between 2 and 4 seconds in
If the control unit 13 determines that there is a forward-backward movement interval between a first point in time and a second point in time in operation S231, the control unit 13 may generate the shortened profile by removing the operation amount over time during the forward-backward movement interval (the interval between 2 and 4 seconds in
The control unit 13 may control the endoscope operating device 12 according to the shortened profile generated as above, thereby preventing a burden to the body of the patient. Further, if the operator finds a correct target site after several trials and errors, by omitting the trials and errors and enabling the endoscope 910 to move directly to the target site along the shortest path, the operation time may be significantly reduced.
Referring to
In operation S232′, the “corrected profile” may be, for example, an operation amount over time that includes a rolling angle variation and a bending angle variation during the forward-backward movement interval and that does not include a translation amount during the forward-backward movement interval.
In operation S233′, the control unit 13 may generate the shortened profile by replacing the operation amount over time during the forward-backward movement interval in the operation profile with the corrected profile.
By this method, even if the moving direction of the end of the endoscope 910 changes during the forward-backward movement interval, the change may be reflected in the driving record of the endoscope 910, and unnecessary repetition of forward and backward translations during the autonomous driving of the endoscope 910 may be prevented. In addition, as shown in
A number of embodiments have been described above. Nevertheless, it should be understood that various modifications may be made to these embodiments. For example, suitable results may be achieved if the described techniques are performed in a different order and/or if components in a described system, architecture, device, or circuit are combined in a different manner and/or replaced or supplemented by other components or their equivalents.
Accordingly, other implementations are within the scope of the following claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2018-0042794 | Apr 2018 | KR | national |
| 10-2019-0043033 | Apr 2019 | KR | national |
This application is a continuation application of U.S. patent application Ser. No. 17/030,331 filed on Sep. 23, 2020, which is a continuation of PCT international application No. PCT/KR2019/004453 filed on Apr. 12, 2019, and claims priority to Korean patent Application No. 10-2018-0042794 filed on Apr. 12, 2018, and Korean patent Application No. 10-2019-0043033 filed on Apr. 12, 2019, the entire disclosures of which are hereby incorporated by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5876325 | Mizuno et al. | Mar 1999 | A |
| 20020022765 | Belson | Feb 2002 | A1 |
| 20070173093 | Kura et al. | Jul 2007 | A1 |
| 20080058836 | Moll | Mar 2008 | A1 |
| 20090287043 | Naito et al. | Nov 2009 | A1 |
| 20100022825 | Yoshie | Jan 2010 | A1 |
| 20110208000 | Honda et al. | Aug 2011 | A1 |
| 20120289777 | Chopra et al. | Nov 2012 | A1 |
| 20120294498 | Popovic | Nov 2012 | A1 |
| 20150230689 | Blohm et al. | Aug 2015 | A1 |
| 20150257847 | Higgins et al. | Sep 2015 | A1 |
| 20160007836 | Kikuchi | Jan 2016 | A1 |
| 20160324399 | Banju et al. | Nov 2016 | A1 |
| 20160360120 | Inoue | Dec 2016 | A1 |
| 20170119412 | Noonan et al. | May 2017 | A1 |
| Number | Date | Country |
|---|---|---|
| 20170107618 | Sep 2017 | KR |
| 2018-0004001 | Jan 2018 | KR |
| Entry |
|---|
| Alfred M. Franz et al., “Electromagnetic Tracking in Medicine—A Review of Technology, Validation, and Applications,” IEEE Transactions on Medical Imaging, vol. 33, Aug. 2014, pp. 1702-1725. |
| Number | Date | Country | |
|---|---|---|---|
| 20220061868 A1 | Mar 2022 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17030331 | Sep 2020 | US |
| Child | 17525549 | US | |
| Parent | PCT/KR2019/004453 | Apr 2019 | US |
| Child | 17030331 | US |
