Patent Grant
8968332
The present invention relates to various embodiments of robotic devices for use in laparoscopic surgery. Specifically, these robotic devices can be inserted into a surgical subject for use in various surgical procedures, providing for performance of various procedures and/or viewing of the area in which a procedure is being performed.
Laparoscopy is minimally invasive surgery (MIS) performed in the abdominal cavity. It has become the treatment of choice for several routinely performed interventions.
However, known laparoscopy technologies are limited in scope and complexity due in part to 1) mobility restrictions resulting from using rigid tools inserted through access ports, and 2) limited visual feedback. That is, long rigid laparoscopic tools inserted through small incisions in the abdomen wall limit the surgeon's range of motion and therefore the complexity of the surgical procedures being performed. Similarly, using a 2-D image from a typically rigid laparoscope inserted through a small incision limits the overall understanding of the surgical environment. Further, current technology requires a third port to accommodate a laparoscope (camera), and each new viewpoint requires an additional incision.
Robotic systems such as the da Vinci® Surgical System (available from Intuitive Surgical, Inc., located in Sunnyvale, Calif.) have been developed to address some of these limitations using stereoscopic vision and more maneuverable end effectors. However, da Vinci® is still restricted by the access ports. Further disadvantages include the size and high cost of the da Vinci® system, the fact that the system is not available in most hospitals and the system's limited sensory and mobility capabilities. In addition, most studies suggest that current robotic systems such as the da Vinci® system offer little or no improvement over standard laparoscopic instruments in the performance of basic skills. See Dakin, G. F. and Gagner, M. (2003) “Comparison of Laparoscopic Skills Performance Between Standard Instruments and Two Surgical Robotic Systems,” Surgical Endoscopy 17: 574-579; Nio, D., Bemelman, W. A., den Boer, K. T., Dunker, M. S., Gouma, D. J., and van Gulik, T. M. (2002) “Efficiency of Manual vs. Robotical (Zeus) Assisted Laparoscopic Surgery in the Performance of Standardized Tasks,” Surgical Endoscopy 16: 412-415; and Melvin, W. S., Needleman, B. J., Krause, K. R., Schneider, C., and Ellison, E. C. (2002) “Computer-Enhanced vs. Standard Laparascopic Antireflux Surgery,” J. Gastrointest Surg 6: 11-16. Further, the da Vinci® system and similar systems are implemented from outside the body and will therefore always be constrained to some degree by the limitations of working through small incisions. For example, these small incisions do not allow the surgeon to view or touch the surgical environment directly, and they constrain the motion of the endpoint of the tools and cameras to arcs of a sphere whose center is the insertion point.
There is a need in the art for improved surgical methods, systems, and devices.
One embodiment disclosed herein is a robotic device having a body, a power source, a connection component, at least one operational arm, and an attachment component. The body is configured to be disposed within a patient. Further, the arm has a first link operably coupled with the body via a first joint and further has an operational component operably coupled with the arm. In addition, the operational arm is not positionable within the body.
According to one alternative embodiment, the arm also has a second link operably coupled with the first link via a second joint. In one implementation, the first joint is a shoulder joint and the second joint is an elbow joint. In accordance with one alternative embodiment, the attachment component is a first magnetic component. In addition, one embodiment of the device has a light component, while another embodiment has a sensor. In one aspect, the sensor is disposed within an interior portion and the body is fluidically sealed whereby no exterior fluids can enter the interior portion.
Another embodiment is a robotic device having a body, a power source, a connection component, a first operational arm, a second operational arm, and an attachment component. The body is configured to be disposed within a patient. The first operational arm has a first link operably coupled with a first end of the body via a first joint, and further has a first operational component operably coupled with the arm. The second operational arm has a second link operably coupled with a second end of the body via a second joint, and further has a second operational component operably coupled with the arm. Neither of the first or second arms are positionable within the body.
In accordance with an alternative implementation, the first operational arm further has a third link operably coupled with the first link via a third joint, and the second operational arm further has a fourth link operably coupled with the second link via a fourth joint. In another embodiment, the device has a sensor positioned between the first and second operational arms. In one aspect, the operational arms and sensor are positioned to substantially approximate a relative configuration of standard laparoscopic tools. Alternatively, the first and second operational arms are configured to substantially approximate movements of standard laparoscopic tools. In one embodiment, the first and second operational components can any of a scalpel, a biopsy tool, a cauterizer, a forceps, a dissector, a clippers, a stapler, an ultrasound probe, a suction component, or an irrigation component.
Another embodiment disclosed herein is a method of surgery. The method includes inserting a robotic device through a natural orifice of a patient and into a passage connected to the natural orifice and creating an incision in a wall of the passage. The method further includes inserting the robotic device into a cavity of the patient and performing a procedure using at least the robotic device. The device has a body, a power source, a connection component, at least one operational arm, and an attachment component. The arm has a first link operably coupled with the body via a first joint and further has an operational component operably coupled with the arm.
In one alternative, the natural orifice is the mouth and the wall is the stomach. Alternatively, the natural orifice is the anus and the wall is the intestinal wall. In a further embodiment, the natural orifice is the umbilicus. According to one implementation, the method includes making only a single incision in the patient. Another embodiment of the method includes positioning the robotic device using a detached handle.
While multiple embodiments are disclosed, still other embodiments will become apparent to those skilled in the art from the following detailed description, which shows and describes illustrative embodiments of the invention. As will be realized, the embodiments disclosed herein are capable of modifications in various obvious aspects, all without departing from the spirit and scope of the various inventions. Accordingly, the drawings and detailed description are to be regarded as illustrative in nature and not restrictive.
The present invention relates to various embodiments of robotic devices for use in surgical methods and systems. Generally, the robotic devices are configured to be inserted into or positioned in a patient's body, such as a body cavity, for example.
The robotic devices fall into three general categories: mobile devices, stationary or “fixed base” devices, and magnetically coupled devices. A “mobile device” includes any robotic device configured to move from one point to another within a patient's body via motive force created by a motor in the device. For example, certain embodiments of mobile devices are capable of traversing abdominal organs in the abdominal cavity. A “fixed base device” is any robotic device that is positioned by a user, such as a surgeon. A “magnetically coupleable device” is any robotic device that can be positioned, operated, or controlled at least in part via a magnet positioned outside the patient's body.
Mobile Robotic Devices
In one alternative embodiment, the device 10 also has a rotation translation component 20 or “tail.” The tail 20 can limit counter-rotation and assist the device 10 in translating the rotation of the wheels 14 into movement from one point to another. The “rotation translation component” is any component or element that assists with the translation or conversion of the wheel rotation into movement of the device. In one embodiment, the tail is spring loaded to retract and thus, according to one embodiment, provide for easy insertion of the robotic device 10 through the entry port of a laparoscopic surgical tool.
In another implementation, the device 10 has no tail 20 and the wired connection component 18 or some other component serves to limit counter-rotation.
Alternatively, a mobile robotic device according to another embodiment can also have one or more operational components (also referred to herein as “manipulators”) and/or one or more sensor components. In these embodiments, the device may or may not have an imaging component. That is, the device can have any combination of one or more imaging components, one or more operational components, and one or more sensor components.
The operational component might be, for example, biopsy graspers. Further, the one or more sensor components could be chosen from, for example, sensors to measure temperature, blood or other tissue or body fluids, humidity, pressure, and/or pH.
In a further alternative, the connection component is a wireless connection component. That is, the controller is wirelessly coupled to, and wirelessly in connection with, the device 10. In such embodiments, the wireless connection component of the device 10 is a transceiver or a transmitter and a receiver to communicate wirelessly with an external component such as a controller. For example,
In accordance with one implementation, a mobile robotic device could be used inside the body of a patient to assist with or perform a surgical procedure. In one aspect, the device is sized to fit through standard laparoscopic tools for use during laparoscopic surgery. In another alternative, the device is sized to be inserted through a natural orifice of the patient, such as the esophagus, as will be described in further detail below. In yet another alternative, the device can be sized and configured in any fashion to be used in surgical procedures.
Any of the several embodiments of mobile robotic devices described herein can be used in any number of ways. For example, one implementation of a mobile robotic device could provide visual feedback with a camera system and tissue dissection or biopsy component with a grasper attached to it. Further, such a robot could also be equipped with a sensor suite that could measure pressure, temperature, pH, humidity, etc.
It is understood that a robotic device as described generally above can take on any known configuration and be equipped with any number of sensors, manipulators, imaging devices, or other known components. That is, a robotic device conforming to certain aspects described herein can, in various embodiments, take on many different configurations, such as cylindrical or spherical shapes, or, alternatively, a shape such as that of a small vehicle, and is not limited to the cylindrical robotic devices depicted in
In addition, as shown in
In one aspect of the invention, the body 32 has a center portion 54 having a radius that is larger than the rest of the body 32. Alternatively, the center portion 54 has the same radius as the rest of the body 32. According to one embodiment, the body 32 can be constructed in any known fashion. For example, according to one embodiment, the body 32 is fabricated via machining or stereolithography.
The device 30 as shown in
In one implementation, the device 30 also has a wireless connection component (not shown) in the form of transmitter and a receiver (not shown) or a transceiver (not shown) for use in a wireless configuration of the device 30 such that any images collected by the camera 38 can be transmitted to an external component for viewing and/or storage of the image and further such that any control signals can be transmitted from an external controller or other external component to the motor 42 and/or other components of the device 30. Alternatively, the device 30 has a wired connection component (not shown) that is attached to the device 30.
In another implementation, the device 30 can also have a light component (not shown) to illuminate the area to be captured by the imaging component. Alternatively, the device 30 has no light component.
According to one embodiment, a robotic device similar to the device 30 depicted in
The device 30 depicted in
According to another embodiment, the robotic device 30 can be constructed without any sharp edges, thereby reducing damage to the patient during use of the device 30. In a further embodiment, the device 30 is comprised of biocompatible materials and/or materials that are easy to sterilize.
A mobile robotic device conforming to certain characteristics of various embodiments discussed herein has a transport component, which is also referred to herein as a “mobility component.” “Transport component” is any component that provides for moving or transporting the device between two points. In one example, the transport component is one or more wheels. For example, the transport components of the mobile robotic devices depicted in
Alternatively, a robotic device as described herein can have any known transport component. That is, the transport component is any known component that allows the device to move from one place to another. The present application contemplates use of alternative methods of mobility such as walking components, treads or tracks (such as used in tanks), hybrid components that include combinations of both wheels and legs, inchworm or snake configurations that move by contorting the body of the device, and the like.
According to one embodiment as depicted in
Each wheel 48, according to one implementation, has a surface texture on its exterior surface as shown in
The raised portion 58, according to one embodiment, defines an outer diameter 58 (doo), while the wheel 48 defines an inner diameter 56 (dr). According to another embodiment, the inner and outer diameters of the wheels in one implementation are 17 mm and 20 mm, respectively. Alternatively, the grouser depth is 1.5 mm, where grouser depth is equal to (doo−dr)/2. In a further alternative, the diameters and/or the grouser depth are any that would be useful for wheels on the mobile devices disclosed herein.
In another embodiment, the helical profile 59 of the wheels has a pitch of 30° as depicted in
In accordance with one implementation, the transport component constitutes at least about 80% of the external surface area of the robotic device. Alternatively, the transport component constitutes at least about 90% of the external surface area of the robotic device. In a further alternative, the transport component constitutes from about 80% to about 98% of the external surface area of the robotic device. In yet another alternative, the transport component constitutes any percentage of the external surface area of the robotic device.
The wheels depicted in
In addition, the wheels depicted in
In certain alternative embodiments, the robotic device has one or more sensor components. In various embodiments, such sensor components include, but are not limited to, sensors to measure or monitor temperature, blood, any other bodily fluids, fluid composition, presence of various gases, such as CO2, for example, or other parameters thereof, humidity, electrical potential, heart rate, respiration rate, humidity, pressure, and/or pH. Further, the one or more sensor components can include one or more imaging components, which shall be considered to be a type of sensor component for purposes of this application. The sensors, including imaging devices, can be any such components or devices known in the art that are compatible with the various designs and configurations of the robotic devices disclosed herein.
According to one embodiment, a robotic device having one or more of the sensors described herein assists the user in the performance of a surgical procedure. In accordance with one implementation, the one or more sensors restore some of the natural monitoring or sensing capabilities that are inherently lost when using standard laparoscopic tools. Thus, the one or more sensor components allow the user to perform more complex procedures and/or more accurately monitor the procedure or the patient.
According to one embodiment, the imaging component can be a camera or any other imaging device. The imaging component can help to increase or improve the view of the area of interest (such as, for example, the area where a procedure will be performed) for the user. According to one embodiment, the imaging component provides real-time video to the user.
Current standard laparoscopes use rigid, single view cameras inserted through a small incision. The camera has a limited field of view and its motion is highly constrained. To obtain a new perspective using this prior art technique often requires the removal and reinsertion of the camera through another incision, increasing patient risk. In contrast to such limited imaging, a robotic device having one or more imaging components according to various embodiments described herein eliminates many of the limitations and disadvantages of standard laparoscopy, providing for an expanded and adjustable field of view with almost unlimited motion, thereby improving the user's visual understanding of the procedural area.
As used herein, the terms “imaging component,” “camera,” and “imaging device” are interchangeable and shall mean the imaging elements and processing circuitry which are used to produce the image signal that travels from the image sensor or collector to a viewing component. According to one embodiment, the image is a moving video image and the viewing component is a standard video viewing component such as a television or video monitor. Alternatively, the image is a still image. In a further alternative, the images are a combination of still and moving video images. The term “image sensor” as used herein means any component that captures images and stores them. In one embodiment, the image sensor is a sensor that stores such images within the structure of each of the pixels in an array of pixels. The terms “signal” or “image signal” as used herein, and unless otherwise more specifically defined, means an image which is found in the form of electrons which have been placed in a specific format or domain. The term “processing circuitry” as used herein refers to the electronic components within the imaging device which receive the image signal from the image sensor and ultimately place the image signal in a usable format. The terms “timing and control circuits” or “circuitry” as used herein refer to the electronic components which control the release of the image signal from the pixel array.
In accordance with one implementation, the imaging component is a small camera. In one exemplary embodiment, the imaging component is a complementary metal oxide semiconductor (“CMOS”) digital image sensor such as Model No. MT9V125 from Micron Technology, Inc., located in Boise, Id. Alternatively, the imaging component is a square 7 mm camera. In an alternative example, the camera can be any small camera similar to those currently used in cellular or mobile phones. In another example, the imaging device can be any imaging device currently used in or with endoscopic devices. In one embodiment, the imaging device is any device that provides a sufficient depth of field to observe the entire abdominal cavity.
According to another embodiment, the imaging device can employ any common solid state image sensor including a charged coupled device (CCD), charge injection device (CID), photo diode array (PDA), or any other CMOS, which offers functionality with simplified system interfacing. For example, a suitable CMOS imager including active pixel-type arrays is disclosed in U.S. Pat. No. 5,471,515, which is hereby incorporated herein by reference in its entirety. This CMOS imager can incorporate a number of other different electronic controls that are usually found on multiple circuit boards of much larger size. For example, timing circuits, and special functions such as zoom and anti-jitter controls can be placed on the same circuit board containing the CMOS pixel array without significantly increasing the overall size of the host circuit board. Alternatively, the imaging device is a CCD/CMOS hybrid available from Suni Microsystems, Inc. in Mountain View, Calif.
In accordance with one implementation, the imaging device provides video output in NTSC format. For example, any commercially-available small NTSC video format transmission chips suitable for the devices described herein can be used. Alternatively, any known video output in any known format can be incorporated into any device described herein.
The imaging component, according to one embodiment, has a manual focus adjustment component. Alternatively, the imaging component has a mechanically-actuated adjustable-focus component. A variety of adjustable-focus mechanisms are known in the art and suitable for actuating focusing of many types of known imaging components.
In one embodiment, the imaging component is capable of focusing in range from about 2 mm to infinity. Alternatively, the imaging component can have a focusing range similar to that of any known adjustable focus camera.
Alternatively, the imaging component has an adjustable-focus mechanism 60 as depicted in
In accordance with another embodiment, the imaging component can be controlled externally to adjust various characteristics relating to image quality. For example, according to one embodiment, one or more of the following can be adjusted by a user: color, white balance, saturation, and/or any other known adjustable characteristic. According to one embodiment, this adjustment capability can provide quality feedback in poor viewing conditions such as, for example, low lighting.
According to one implementation, any mobile imaging device disclosed herein can have any known lens that can be used with such devices. In one particular embodiment, the lens is model no. DSL756A, a plastic lens available from Sunex, located in Carlsbad, Calif. This embodiment provides only a short depth of field, which requires adjustable-focus capability. To attain this, the lens of this implementation is attached to an actuation mechanism to provide adjustable focus capability. The lens is moved by the actuation mechanism to provide a range of focus from 2 mm to infinity. Alternatively, the lens can be any lens that can be incorporated into any of the imaging devices described herein.
In a further alternative, the imaging component can include an image stabilization component. For example, according to one embodiment, the device could include on-board accelerometer measurements with image motion estimates derived from optical flow to yield base motion estimates, such as are known in the art. Alternatively, the image stabilization component can be any such commercially-available component. Optical flow has been shown to yield reliable estimates of displacements computed across successive image frames. Using these robot base motion estimates, image stabilization algorithm can be used to provide image stabilization. Alternatively, any known image stabilization technology can be incorporated for use with the imaging component.
In certain embodiments, the camera is fixed with respect to the body of the robotic device, such that the position of the robot must be changed in order to change the area to be viewed. Alternatively, the camera position can be changed with respect to the device such that the user can move the camera with respect to the robotic device. According to one embodiment, the user controls the position of the camera using a controller that is operably coupled to the device as described in further detail herein.
The robotic device can also, according to one embodiment, have a lighting component to light the area to be viewed. In one example, the lighting component is an LED light. Alternatively, the lighting component can be any illumination source.
According to one implementation, the camera is disposed on the center portion of the body of the device, as shown in
According to one embodiment, the robotic device has one or more operational components. The “operational component,” as used herein, is intended to mean any component that performs some action or procedure related to a surgical or exploratory procedure. According to one embodiment, the operational component is also referred to as a “manipulator” and can be a clamp, scalpel, any type of biopsy tool, a grasper, forceps, stapler, cutting device, cauterizing device, ultrasonic burning device, or other similar component, as set forth in further detail herein. In yet another embodiment, the operational component is any device that can perform, or assist in the performance of, any known surgical or exploratory laparoscopic procedure. In one aspect, the one or more operational components assist with procedures requiring high dexterity. In currently known techniques, movement is restricted, as passing the rigid laparoscopic tool through a small incision restricts movement and positioning of the tool tip. In contrast, a robotic device having an operational component inside a cavity is not subject to the same constraints.
In one implementation, the operational component can also include an arm or other positioning component. For example, the operational component can include an arm and a biopsy tool. Alternatively, the operational component can include a positioning component and any operational component as described above.
According to one embodiment, any operational component described or contemplated herein can be an off-the-shelf surgical tool or modified version thereof. Alternatively, any such operational component can be constructed de novo.
The operational component depicted in
The joints 84 are configured in any known fashion. In one example as depicted in
In one implementation, the arm was constructed using stereolithography. According to one embodiment, stereolithography can be used to construct the linkages and the base section out of a cured resin material similar to plastic.
The motor, according to one embodiment, that can be used in the linkages is a DC micromotor with encoders manufactured by MicroMo Electronics, located in Clearwater, Fla. The motor is a 6 V motor having a 15,800 rpm no-load speed, 0.057 oz-in stall torque, and weighed 0.12 oz. The motor has an 8 mm diameter and is 16 mm long. Due to its high no-load speed, a precision planetary gearhead is used. Further description of the motor, gearhead, and an encoder that can be used with the motor are described in Example 2. Alternatively, the arm can use a low voltage motor, such as a 3 V motor.
In one implementation, the arm has an encoder used for the indication and control of both shaft velocity and the direction of rotation, as well as for positioning. In one embodiment, the encoder is a 10 mm magnetic encoder. It is 16.5 mm long, but only adds 11.5 mm to the total length of the assembly.
In one embodiment as depicted in
In one embodiment, the manipulator is a biopsy forceps or grasper. According to one aspect, the manipulator includes a biopsy forceps or graspers at one end of an arm.
In another embodiment, the manipulator of the present invention includes an actuation mechanism that generates forces required for operating the manipulator. For example, according to one embodiment in which the manipulator is a biopsy forceps or graspers, the manipulator also has an actuation mechanism that generates sufficient force to allow the forceps or graspers to cut/obtain a biopsy sample. According to one embodiment, the actuation mechanism generates a drawbar force of magnitude greater than 0.6 N. Alternatively, the actuation mechanism generates any amount of force sufficient to obtain a biopsy sample. In a further alternative, the actuation mechanism generates a sufficient force to operate any type of manipulator, such as a clamp, stapler, cutter, cauterizer, burner, etc.
In one embodiment, the body 104 also contains an imaging component (not shown), camera lens 108, motor and video control boards (not shown), and actuation motor (not shown) and a mechanism for camera adjustable-focus (not shown). In this embodiment, the imaging component and lens 108 are offset to the side to allow space for the biopsy grasper 102. The wheel 110 on the camera side has slots 112 machined in it to allow for space for the camera lens 108 to see the abdominal environment and the biopsy grasper 102. Alternatively, the camera and lens 108 are disposed anywhere on the robotic device 100 such that the camera can be used to view the surgical area and/or the biopsy grasper 102 during use. The device 100 a wired connection component 114 that is connected to an external component (not shown).
In use, a robotic device with a camera and a biopsy tool such as the devices depicted in
In an alternative embodiment, the manipulator is a drug delivery component. That is, according to one implementation, robotic devices disclosed herein can have a drug delivery component or system that delivers an agent to an animal, including a human. In one embodiment, the agent is a hemostatic agent. Alternatively, the agent can be any deliverable composition for delivery to an animal, including a human.
In one embodiment, the dual reservoirs 162 of
According to one embodiment, the spring-loaded catch lever 176 is a shape memory alloy and is actuated with a SMA wire trigger. SMA wires are made of a nickel-titanium alloy that is easily stretched at room temperature. However, as the wires are heated by passing an electric current through them, they shorten in length and exert a force that is greater than the force required to stretch them. In one embodiment, the wires shorten in length by up to approximately 8% and exert approximately 5 times the force required to stretch them.
A further alternative embodiment of the actuator mechanism is depicted in
Alternatively, the actuator mechanism can be any known device for providing for linear displacement of the reservoir plungers 180 that dispense the agent. According to one implementation, the actuator ensures uniform delivery of the agent from the storage reservoir(s).
Alternatively, the mixing component is any known component for mixing two agents, including, but not limited to, hemostatic agents, that can implemented with one or more of the robotic devices described herein.
In accordance with one aspect, the reservoir or reservoirs have at least one externally accessible loading port configured to allow for loading, injecting, or otherwise placing the agent or components into the reservoir. The loading port is a standard rubber stopper and seal commonly used for vaccine vials. Such a rubber stopper and seal facilitates transfer of any agent using a standard syringe. Alternatively, the loading port is any known type of loading port of any known configuration. According to one embodiment, such a loading port is useful for known agents that must be reconstituted shortly before use, such as on-site reconstitution. As such, the loading port or ports accommodate the need for on-site loading of the compounds.
According to one aspect, any robotic device embodiment described herein is connected to an external controller via a connection component. According to one embodiment, the connection component is a wire, cord, or other physical flexible coupling. For purposes of this application, the physical or “wired” connection component is also referred to as “tethered” or “a tether.” The flexible connection component can be any component that is coupled at one end to the robotic device and is flexible, pliable, or otherwise capable of being easily formed or manipulated into different shapes or configurations. According to one embodiment, the connection component includes one or more wires or cords or any other type of component operably coupled at the second end to an external unit or device. The component in this embodiment is configured to transmit or convey power and/or data, or anything else necessary or useful for operation of the device between the robotic unit and the external unit or device. In a further alternative, the connection component comprises at least two wires or cords or other such components, each of which are connected to a separate external unit (which, in one example, are a power source and a data transmission and receiver unit as described below).
Alternatively, the connection component is a wireless connection component. That is, the robotic device communicates wirelessly with a controller or any other external component. The wireless coupling is also referred to herein as “untethered.” An “untethered device” or “wireless device” is intended for purposes of this application to mean any device that is fully enclosed within the body such that no portion of the device is external to the body for at least a portion of the surgical procedure or, alternatively, any device that operates within the body while the device is not physically connected to any external object for at least a portion of the surgical procedure. In one embodiment, an untethered robotic device transmits and receives data wirelessly, including data required for controlling the device. In this embodiment, the robotic device has an internal power supply, along with a receiver and transmitter for wireless connection.
The receiver and transmitter used with a wireless robotic device as described herein can be any known receiver and transmitter. For example, any known receiver and/or transmitter used in remote vehicle locking devices, remote controls, mobile phones.
In one embodiment, the data or information transmitted to the robotic device could include user command signals for controlling the device, such as signals to move or otherwise operate various components. According to one implementation, the data or information transmitted from the robotic device to an external component/unit could include data from the imaging component or any sensors. Alternatively, the data or information transmitted between the device and any external component/unit can be any data or information that may be useful in the operation of the device.
According to another implementation, any robotic device embodiment described herein is connected via a connection component not only to the external controller, but also to one or more other robotic devices, such devices being either as described herein or otherwise known in the art. That is, according to one embodiment, two or more robotic devices can be operably coupled to each other as well as an external unit or device. According to one embodiment in which there are two robotic devices, the two devices are operably coupled to each other and an external unit or device by a flexible connection component. That is, the two devices are operably coupled to each other by a flexible connection component that is coupled to each device and each device is also operably coupled to an external unit or device by a flexible connection component. In one embodiment, there are three separate flexible connection components: (1) a connection component connecting the two robotic devices, (2) a connection component connecting one of the robotic devices to the external unit, and (3) a connection component connecting the other of the robotic devices to the external unit. Alternatively, one connection component is operably coupled to both devices and the external unit. In a further alternative, any number of connection components can be used in any configuration to provide for connection of two robotic devices to each other and an external unit.
Alternatively, the two or more robotic devices are operably coupled to each other as well as an external unit or device in an untethered fashion. That is, the robotic devices are operably coupled to each other and an external unit or device in a fashion such that they are not physically connected. In one embodiment, the devices and the external unit are operably coupled wirelessly.
In one aspect, any robotic device described herein has a drive component. The “drive component,” as defined herein, is any component configured to provide motive force such that the robotic device can move from one place to another or some component or piece of the robotic device can move, including any such component as described herein. The drive component is also referred to herein as an “actuator.” In one implementation, the drive component is a motor.
The actuator can be chosen from any number of different actuators. For example, one actuator that can be incorporated into many, if not all, of the robotic devices described herein, is a brushless direct current motor, such as, for example, model no. SBLO4-0829 with gearhead PG04-337 (available from Namiki Precision of California, which is located in Belmont, Calif.). According to one embodiment, this motor requires external connection, which is generally provided by a circuit supplied by the manufacturer. In another implementation, the motor is model no. SBL02-06H1 with gearhead PG02-337, also available from Namiki.
Alternatively, any brushless direct current motor can be used. In a further alternative, another motor that can be used to operate various components of a robotic device, such as a manipulator, is a permanent magnet DC motor made by MicroMo™ Electronics, Inc. (located in Clearwater, Fla.). In yet another alternative, any known permanent magnet DC motors can be used with the robotic devices described herein.
The motor runs on a nominal 3 V and can provide 10.6 [mNm] stall torque at 80 rpm. This motor provides a design factor of 4 for the robot on a 75-degree slope (if frictional force is sufficient to prevent sliding).
In addition, other actuators that can be used with the robotic devices described herein include shape memory alloys, piezoelectric-based actuators, pneumatic motors, hydraulic motors, or the like. Alternatively, the robotic devices described herein can use any type of compatible actuator.
According to one embodiment, the actuator can have a control component, also referred to as a “control board.” The control board can have a potentiometer that controls the speed of the motor. relationship between the terminals that created the voltage divider. According to one embodiment, the control board can also control the direction of the motor's rotation.
In accordance with one implementation, any robotic device as described herein can have an external control component, also referred to herein as a “controller.” That is, at least some of the devices herein are operated by a controller that is positioned at a location external to the animal or human.
In one embodiment, the external control component transmits and/or receives data. In one example, the unit is a controller unit configured to control the operation of the robotic device by transmitting data such as electronic operational instructions via the connection component, wherein the connection component can be a wired or physical component or a wireless component. The data transmitted or conveyed by the connection component can also include, but is not limited to, electronic data collected by the device such as electronic photographs or biopsy data or any other type of data collected by the device. Alternatively, the external unit is any component, device, or unit that can be used to transmit or receive data.
According to one embodiment, the external component is a joystick controller. In another example, the external component is any component, device, or unit that can be used to control or operate the robotic device, such as a touch screen, a keyboard, a steering wheel, a button or set of buttons, or any other known control device. Further, the external component can also be a controller that is actuated by voice, such as a voice activation component. Further, a controller may be purchased from commercial sources, constructed de novo, or commercially available controllers may be customized to control any robotic device or any robotic device components disclosed herein.
In one example, the controller includes the “thumb sticks” from a Playstation™ Dual-Shock controller. In this example, the Playstation™ controller had two analog thumb sticks, each with two degrees of freedom. This allows the operator to move the thumbsticks a finite amount in an XY coordinate plane such that pushing the stick forward a little yields a different output than pushing the stick forward a great deal. That is, the thumb sticks provide speed control such that movement can be sped up or slowed down based on the amount that the stick is pushed in the corresponding direction.
According to one embodiment, the connections between the controller and the robotic device are configured such that each wheel is controlled by a separate joystick.
In another example, the controller is a directional pad similar to the directional pad on an original Nintendo™ game system. The pad resembles a + sign and has four discrete directions.
In use, the controller can be used to control the movement of the robotic device and further to control the operation of any components of the device such as a sensor component, a manipulator component, or any other such component. For example, one embodiment of the controller controls the wheels, the focus adjustment of the camera, and further controls the biopsy tool.
In accordance with one embodiment, the control component also serves as a power source for the robotic device.
In accordance with one embodiment, a mobile robotic device is coupled to an image display component. Signal from the camera is transmitted in any format (e.g., NTSC, digital, PAL, etc.) to the image display component. According to one embodiment, the signal is a video signal or a still image signal. In one embodiment, the image display component is a video display that can be viewed by the operator. Alternatively, the image display component is a still image display. In a further alternative, the image display component displays video and still images. In one embodiment, the image display component is a standard video monitor. Those of ordinary skill in the art recognize that a signal from a camera can be processed to produce a display signal for many different types of display devices, including televisions configured to display an NTSC signal, televisions configured to display a PAL signal, cathode ray tube based computer monitors, LCD monitors, and plasma displays. In a further embodiment, the image display component is any known image display component capable of displaying the images collected by a camera that can be used with any of the robotic devices described herein.
In one embodiment, the image display component is a component of the controller.
A robotic device as described herein, according to one implementation, has a power source or power supply. According to one embodiment, the power source is integrated into the body of robotic device. In this embodiment, the power source can be one or more batteries. The battery can be an alkaline, lithium, nickel-cadmium, or any other type of battery known in the art.
Alternatively, the power source is positioned in a location external to the body of the patient. In this embodiment, the connection component operably coupled to the power source and the robotic device transmits or conveys power between the power source and the robotic device. For example, the external power source according to one embodiment is an electrical power source such as a battery or any other source of electricity. In this example, the electricity is conveyed from the battery to the robotic device via the connection component, which is any known wire or cord configured to convey electricity, and thereby supplies power to the robotic device, including the motor of the robotic device. In one example, the power source is integrated into the control component or is operably coupled to the control component.
According to one embodiment, the power source can be any battery as described above. Alternatively, the power source can be magnetic induction, piezoelectrics, nuclear, fluid dynamic, solar or any other known power source that can be used to supply power to any robotic device described herein.
Fixed Base Devices
Certain robotic devices disclosed herein relate to fixed base robots. As discussed above, a “fixed base robotic device” is any robotic device that has no propelled transport component or is positioned manually by a user. Such a device is also referred to herein as a “stationary” robotic device. In one embodiment, a fixed base robot has a camera and is positioned manually by the user to provide visual feedback or a visual overview of the target area. A fixed base robotic camera device according to one implementation facilitates the application of laparoscopy and other surgical techniques by providing a remote-control camera robot to provide visual feedback during a surgical procedure, thereby minimizing incisions and patient risk.
In one embodiment, the device 220 is made of a biocompatible material capable of being easily sterilized. According to one embodiment, the materials can include, but are not limited to, sterilizable plastics and/or metals. Alternatively, the device 220 can be made of any material that can be used in surgical procedures.
The body 222 can take on many different configurations, such as cylindrical or spherical shapes so as to be compatible with laparoscopic tools known currently in the art. However, as with the other components, the body 222 configuration is not limited to that exemplified herein. In general, the only constraints on the shape of the body are that the body be able to incorporate at least one of the components described herein.
The handle 232, according to one embodiment as depicted in
The light component 226, according to one embodiment, is configured to light the area to be viewed, also referred to as the “field of view.” In one implementation, the light component 226 is proximate to the imaging component to provide constant or variable illumination for the camera. Alternatively, the light component 226 is associated with the handle 232 as depicted in
In one example, the lighting component 226 is an LED light. Alternatively, an exemplary light source is two 5 mm LEDs. In a further alternative, the lighting component 226 can be any suitable illumination source.
In one implementation, the imaging component 224 depicted in
The imaging component can help to increase or improve the view of the area of interest (such as, for example, the area where a procedure will be performed) for the user. According to one embodiment, the imaging component provides real-time video to the user. Alternatively, the imaging component can be any imaging component as described above with respect to the mobile robotic devices.
In accordance with one implementation, the tilting component 242 is pivotally coupled to the body 248 via a pin (not shown). Alternatively, the tilting component can be a standard ratchet mechanism or any other type of suitable component known in the art. According to one embodiment, the tilting component 242 can tilt up to about 45 degrees from vertical (i.e., a range of about 90 degrees). Alternatively, the tilting component 242 can tilt any amount ranging from about 0 degrees to about 360 degrees from vertical, or the tilting component 242 can configured to rotate beyond 360 degrees or can rotate multiple times. In certain embodiments such as the embodiment depicted in
The panning component 244, 246, according to one embodiment, has the two components 244, 246 that rotate with respect to each other as described above with respect to
In one aspect, any fixed base robotic device described herein has a drive component (not shown). In accordance with certain embodiments, the fixed base robotic device can have more than one drive component. For example, in one embodiment, a fixed base robotic device has a motor for actuating the panning component and another motor for actuating the tilting component. Such motors can be housed in the body component and/or the support component. In one example, the actuator or actuators are independent permanent magnet DC motors available from MicroMo™ Electronics, Inc. in Clearwater, Fla. Other suitable actuators include shape memory alloys, piezoelectric-based actuators, pneumatic motors, hydraulic motors, or the like. Alternatively, the drive component can be any drive component as described in detail above with respect to mobile robotic devices. In a further alternative embodiment, the panning and tilting components can be actuated manually.
In one embodiment, the actuator is coupled to a standard rotary-to-translatory coupling such as a lead screw, a gear, or a pulley. In this fashion, the force created by the actuator is translated with the rotary-to translatory coupling.
Moreover, it is also contemplated that the body or camera in certain embodiments could be capable of a side-to-side motion (e.g., yaw).
Various embodiments of fixed base robotic devices have an adjustable-focus component. For example, one embodiment of an adjustable-focus component 60 that can incorporated into various embodiments of the fixed base robotic devices described herein is depicted in
According to one embodiment, the imaging component can have a lens cleaning component. For example, the lens cleaning component can be a wiper blade or sacrificial film compose of multiple layers for maintaining a clear view of the target environment. In a further embodiment, the lens cleaning component can be any known mechanism or component for cleaning a camera lens.
Certain embodiments of the fixed base robotic devices, such as the embodiment depicted in
The support component 266, as depicted in
According to one aspect, any fixed base robotic device embodiment described herein is connected to an external controller via a connection component. According to one embodiment, the connection component is any wired or flexible connection component embodiment or configuration as described above with respect to mobile robotic devices. Alternatively, the connection component is a wireless connection component according to any embodiment or configuration as described above with respect to mobile robotic devices. The receiver and transmitter used with a wireless robotic device as described herein can be any known receiver and transmitter, as also described above. According to another implementation described in additional detail above with respect to the mobile devices, any fixed base robotic device embodiment described herein can be connected via a (wired or wireless) connection component not only to the external controller, but also to one or more other robotic devices of any type or configuration, such devices being either as described herein or otherwise known in the art.
In one embodiment, the data or information transmitted to the robotic device could include user command signals for controlling the device, such as signals to move or otherwise operate various components. According to one implementation, the data or information transmitted from the robotic device to an external component/unit could include data from the imaging component or any sensors. Alternatively, the data or information transmitted between the device and any external component/unit can be any data or information that may be useful in the operation of the device.
In accordance with one implementation, any fixed base robotic device as described herein can have an external control component according to any embodiment as described above with respect to the mobile robotic devices. That is, at least some of the fixed base devices herein are operated by a controller that is positioned at a location external to the animal or human. In one embodiment, the external control component transmits and/or receives data. In one example, the unit is a controller unit configured to control the operation of the robotic device by transmitting data such as electronic operational instructions via the connection component, wherein the connection component can be a wired or physical component or a wireless component. Alternatively, the external unit is any component, device, or unit that can be used to transmit or receive data.
In use, the controller can be used to control the movement or operation of any components of the device such as the camera component, a sensor component, or any other component. For example, one embodiment of the controller controls the focus adjustment of the camera, and further controls the panning and/or tilting functions of the device.
According to one embodiment, the control component is configured to control the operation of the image sensor, the panning component, and the tilting component. In one embodiment, the control component transmits signals containing operational instructions relating to controlling each of those components, such as, for example, signals containing operational instructions to the image sensor relating to image quality adjustment, etc.
In accordance with one embodiment, the control component also serves as a power source for the robotic device.
According to one implementation, the fixed base robotic device is coupled to an image display component. The image display component can be any image display component as described above with respect to the mobile robotic devices.
A fixed base robotic device as described herein, according to one implementation, has a power source or power supply. According to one embodiment, the power source is any power source having any configuration as described above with respect to the mobile robotic devices. According to various embodiments, power can be provided by an external tether or an internal power source. When the device is wireless (that is, the connection component is wireless), an internal power supply can be used. Various implementations of the fixed base robotic devices can use alkaline, lithium, nickel-cadmium, or any other type of battery known in the art. Alternatively, the power source can be magnetic induction, piezoelectrics, fluid dynamics, solar power, or any other known power source. In a further alternative, the power source is a power unit positioned within the patient's body. In this embodiment, the power unit can be used to supply power not only to one or more robotic camera devices, but can also to any other surgical robotic devices.
In one embodiment, the fixed base robotic device has one or more sensor components. In various embodiments, such sensor components include any of the sensor components as described above with respect to the mobile robotic devices.
According to one embodiment, any of the components on any fixed base robotic device as described herein can be known, commercially available components.
In use, any of the fixed base robotic devices can be used in various surgical procedures. For example, a fixed base device can be used in combination with a laparoscopic surgical tool, wherein the device is adapted to fit through a port of the laparoscopic surgical tool and used for obtaining an internal image of an animal. In still other embodiments, the whole of the device is introduced into an open space to obtain internal images.
Alternatively, the fixed base robotic devices can be used in oral surgery and general dental procedures to provide an image of particularly difficult-to-access locations. Additionally, it will also be appreciated by those skilled in the art that the devices set forth herein can be applied to other functional disciplines wherein the device can be used to view difficult-to-access locations for industrial equipment and the like. For example, the device could be used to replace many industrial boroscopes.
Magnetically Coupleable Robotic Devices and Systems
Certain robotic devices disclosed herein relate to magnetically coupleable robotic devices and related systems. As discussed above, a “magnetically coupleable device” is any robotic device that can be positioned, operated, or controlled at least in part via a magnet positioned outside the patient's body.
In one implementation, the robotic device 302 is a cylindrical robotic device 302 having an imaging component 306 and a lighting component 308, along with two magnets 310, 312, each positioned at an end of the device 302. In accordance with one embodiment, the device magnets 310, 312 are magnetically coupled with magnets 314, 316 on the handle 304 such that the device 302 is urged toward and held against the body cavity wall 320. In one embodiment, the magnets 310, 312 are configured to ensure that the imaging component 306 is positioned to provide a view of the body cavity or the target area of interest. Alternatively, the robotic device can be any known robotic device as disclosed herein or otherwise known in the art that can be positioned, operated, or controlled at least in part by an external magnet.
The imaging component 306, according to one embodiment is a single camera. Alternatively, the imaging component 306 can be multiple cameras used to create stereoscopic vision.
It is understood that the magnets 310, 312 can be positioned anywhere in or on the device 302. It is also understood that the device 302 can have two magnets 310, 312, one disposed at each end of the device 302 as shown in
Similarly, it is understood that the magnets 314, 316 in the handle 304 can be positioned anywhere in or on the handle 304 so long as the magnets can be magnetically coupleable with the magnets in the device. It is also understood that the handle 304 can have two magnets 314, 316 as shown in
In accordance with one aspect, the magnetic handle 304, also referred to herein as an “external magnet”) is in the shape of a handle. It is understood, however, that “magnetic handle” and/or “external magnet” as used herein is intended to encompass any magnetic component that is magnetically coupleable with any robotic device as described herein such that the magnetic component can be used to position, operate, or control the device.
In one embodiment, the handle 304 can be rotated as shown by arrow 318 to allow a tilting functionality for the imaging component 306. That is, the imaging component 306 can “tilt,” which shall mean, for purposes of the present application, moving perpendicular to the axis of the cylinder of the device 302. Further, the device 302 can also provide for a panning functionality via rotation of the imaging component 306 as shown by arrow 322, as described in further detail below. That is, the imaging component 306 can also “pan,” which shall mean, for purposes of the present application, rotating about the axis of the cylinder.
In use, the handle 304 can be moved across the entire abdomen to a desired position by moving the handle 304 outside the body. Alternatively, the device 302 can be positioned anywhere within an animal body and positioned, operated, or controlled at least in part by the magnetic handle 304 positioned outside the body. According to one implementation, the device 302 can also reattach itself if one end is knocked free. In one embodiment, the magnets 310, 312 provide sufficient magnetic attraction with the external magnet to resist vibration. Use of magnets allows for easy adjustment via the handle 304 outside the abdomen and easy attachment to the wall after insertion. In another embodiment, attachment is achieved by placing the handle 304 against the abdomen near the entry incision and pressing the handle 304 inward. The opposing poles of the magnets cause the device 302 to be lifted to the abdominal wall.
In one embodiment, the device 302 is sized to be inserted into the abdominal cavity and can be positioned on the abdominal wall such that it does not obstruct any surgical operation or procedure being performed. In such an embodiment, the imaging component 306 provides a view of the surgical procedure for the user. In one variation of this embodiment, the device 302 is sized to fit through standard laparoscopic tools.
In accordance with one implementation, an inner body similar to the body 360 depicted in
According to one embodiment, the adjustable focusing mechanism 366 includes two coils of wire (not shown) and a magnetic field produced by two additional magnets (not shown) near the lens 368. Current through the coiled wire that is placed in magnetic field creates a force that is used to drive the position of the lens 368. In one embodiment, a restoring force is provided that urges the lens back to its resting position when the force from the coiled wire is removed. According to one implementation, the restoring force is provided by a foam component. Alternatively, any known component for providing a restoring force can be used.
The lens 384 is operably coupled to a lens adjustment component 390 and the mirror 386 is operably coupled to a mirror adjustment component 392. Light is allowed through the aperture 388 and reflected off the mirror 386 up to the imaging component 382 through the lens 384. In this embodiment, adjusting the angle of the mirror 386 makes it possible to capture an image from a wide variety of different angles without otherwise tilting the device 380. In this embodiment, the mirror adjustment component 392 includes a 6 mm motor that operates to turn a threaded rod to move a nut up and down in a guide slot. The nut is attached to the mirror causing it to change its tilt angle. Alternatively, any known mechanism for providing adjustment of the disposition of the mirror 386 can be used. In one embodiment, adjustable mirror 386 allows for the capture of images from a wide area around the device 380. That is, the device 380 can remain relatively stationary.
According to one embodiment, the image is focused by moving the lens 384. In this embodiment, lens 384 adjustment is accomplished with the lens adjustment component 390. The component 390 has an actuation component operably coupled to a threaded rod that drives a nut in a guide slot, where the lens is rigidly fixed to the nut. According to an alternative embodiment, focusing is accomplished by any known focusing component.
According to one embodiment, the bottom portion 402 is a solid portion with cavities for the actuation component 394 and, according to another embodiment, the lens adjustment motor and the mirror adjustment motor.
In this embodiment, the device 380 provides for panning the imaging component 382 by rotating the device 380 using the actuation component 394 and further provides for tilting functionality via tilting the mirror 386 as described above.
Alternatively, the magnetically coupleable robotic device can have any known component that provides for panning capabilities and/or any known component that provides for tilting capabilities. In another embodiment, the device has no panning capabilities and/or no tilting capabilities. In a further embodiment, the device has both pan and tilt components.
In one implementation, the cylindrical housing 422 has an imaging component (not shown). According to one implementation, the imaging component is a camera. Alternatively, the imaging component is a pair of stereoscopic cameras.
The device 420, according to one implementation, has an actuator (not shown) for actuating each of the joints 426. In one embodiment, the device 420 has a separate actuator for each joint 426. Alternatively, the device 420 has one or more actuators. In one embodiment, each actuator is disposed within an arm 424. Alternatively, each actuator is disposed in any portion of the device 420.
The device 420, according to one aspect, has an external controller (not shown) coupled to the device 420. The controller can be coupled to the device 420 via a wired connection component or it can be coupled wirelessly. In certain embodiments, the controller can be any controller as described above with respect to other embodiments of robotic devices. In another embodiment, the controller is a controller similar to those used in industrial robots in which each joint is controlled or activated separately using a switch or button or other type of input component (certain versions of such a controller also being referred to in the art as a “teach pendant”). Alternatively, the controller is a joystick controller similar to those described above.
In a further alternative, the controller is a “closed loop” controller system commonly used in robotic technologies. As is understood, a “closed loop” controller system is a system that provides for a controller that allows the user to provide specific instructions regarding a specific movement or action and further provides for a feedback sensor that ensures the device completes the specific movement or action. This system allows for very specific instructions or commands and very precise actions. For example, in the embodiment in
In yet another alternative, the controller is a component having a configuration similar to the device component itself. For example, in the embodiment depicted in
The controller is used to activate the components of the device 420. That is, the controller can be operated by a user to operate the device 420. The controller is coupled to the actuators (not shown) of the device 420 to operate the arms 424 and joints 426, any imaging component, and any operational components operably coupled to the device 420. Alternatively, two or more controllers (not shown) can be coupled to the device 420 to operate different components of the device 420.
In use, the robotic device 420 is a retractor device 420, according to one embodiment. The device 420 can be inserted into a patient's body while in the linear configuration of
In one embodiment, the device is controlled by an operator to provide gross tissue manipulation, stereoscopic vision and visual feedback via the imaging component, and/or task assistance capabilities for any type of procedure within a patient's body. That is, once the device 420 has been positioned inside the body, the user can operate an external controller to activate the actuators to configure the arms 424 into an appropriate configuration. In one embodiment, the device 420 is used for gross manipulation of tissue and organs, retracting those that physically or visually obstruct the surgeon. In this embodiment, the arms 424 of the device 420 can be used to hold back tissue and organs to allow the surgeon physical and visual access to the necessary surgical field.
According to one embodiment, the positioning or configuration of the arms 424 can be maintained following initial positioning by the user such that the user does not need to rely on clamping or manual holding. In addition, the configuration of the arms 424 can be remotely adjusted throughout the procedure by the user.
In an alternative embodiment, a magnetically coupleable device can have additional components and be used for additional procedures. That is, the device can have at least one operational component attached to an arm or the cylindrical housing.
In one embodiment, the forceps 450 are similar to standard hand-held laparoscopic forceps, similar to the forceps tool 480 depicted in
In one implementation, the cautery 452 can be a commercially-available handheld single use cautery tools such as those made by ACMI Corporation, Medtronic, or several other manufacturers. Such devices consist of a specialized tip and often use two standard AA batteries as a power source. The devices generally operate at 3 volts and pass approximately 2 amps through the tip to reach temperatures around 1200° C. (2200° F.). The tips of these devices can be removed and installed as detachable operational components. In one embodiment, the cautery tool also has a Darlington transistor pair that is controlled by a microprocessor, and through which electrical current can be passed. Alternatively, the cautery component 452 can be any known component that can be used with a magnetically coupleable robotic device of any embodiment described herein.
Alternatively, the operational component according can be a grasper or a scalpel. In a further embodiment, the operational component can be any operational component as described above with respect to the mobile robotic device embodiments that could be used with the present magnetically coupleable robotic device. For example, the operational component can be a dissector, a clippers, a stapler, an ultrasound probe, a suction component, an irrigation component, or any component that may be useful in a medical procedure of any kind. As such, a magnetically coupleable device as described herein with the operational component could be used in such procedures as tissue dissection, suturing, or any other medical procedure that could be performed with an operational component coupled to a magnetically coupleable device as described herein.
In one embodiment, the joints 446A, 446B depicted in
As shown in
In one embodiment, the imaging component 454 includes a CMOS sensor available from by Micron Technology, Inc., located in Boise, Id. The sensor consists of an array of 640×480 pixels with an active image area of 3.63 mm×2.78 mm, and has on-board signal processing circuitry that outputs an analog color NTSC composite video signal. The sensor also has several settings that can be used to optimize image quality. These are programmable via a standard serial connection, and include color saturation, brightness, hue, white balance, exposure, and gain. The entire sensor is 9 mm×9 mm×1.3 mm in size, requires only a single-ended 2.5 Volt power supply, and draws approximately 40 mA (100 mW). Alternatively, any known imaging component can be used. According to another embodiment, any one of a number of compound lenses matched to these types of sensors are widely available. In addition, the device 440 can also have a variable focus mechanism based on a voice coil design. Alternatively, any known variable focus component can be used.
In accordance with one implementation, the imaging component can provide visual feedback relating to the operation of the device 420. For example, the imaging component can be used to determine the location of the arms 424 and/or provide visual feedback to the user with respect to any surgical procedure being performed. That is, the user could utilize the visual feedback from the imaging component to aid in positioning of tissues for inspection or in the performance of any procedure that might be accomplished with an operational component, such as dissection or suturing. All of this type of information can be utilized for the adjustment of the arms 424 to attain any desired configuration for providing tissue retraction or procedural assistance.
In one aspect, the device 440 as configured in
In one embodiment, each actuator has two 6 mm brushed motors and two springs disposed in a cylindrical arm 424. The actuator articulates a joint 426 primarily in two planes. In this embodiment, the rotational motion of the motor is transformed to linear motion using a lead screw and nut in a guide. Each nut is attached via a swing or cable to one side of the joint 426. The motor pulls this segment of the joint 426 causing the joint 426 to rotate. A spring attached to the other side of the joint 426 provides the restoring force for articulation of the joint 426 in one plane. Alternatively, the actuator can be any known actuation component that can be used with this device 420.
The arms 470 are movably coupled to the housing 467 to allow for movement of the arms 470. More specifically, in the embodiment depicted in
The two operational components 468, 469 are each coupled to an arm 470 such that each operational component 468, 469 can move in relation to the respective arm 470. More specifically, in this embodiment, both operational components 468, 469 are movably coupled to the arms 470 such that each of the components 468, 469 can extend and retract laterally along the axis of the arms 470 as depicted by the arrow 474. Further, the component 468, 469 can also rotate around that axis as indicated by the arrow 475. It is understood that any known joint, rotatable component, or any other coupling component can be used to couple the components 468, 469 to the arms 470 such that the arms components 468, 469 can move in relation to the arms 470. In addition, according to an alternative embodiment, the components 468, 469 are coupled to a second set of arms (not shown) that are movably coupled to the arms 470 such that the second set of arms can be moved laterally (arrow 474) and/or rotationally (arrow 475). In further embodiments, the second set of arms can each have a single motion or multi-motion joint on its distal end that is operably coupled to the operational component whereby the operational component can be move in relation to the second set of arms.
The device 466, according to one aspect, has a flat surface (not shown) along the side of the housing 467 opposite the imaging component 472. When the device 466 is magnetically coupled via the magnet component 473 to an exterior magnet and thus positioned against an interior surface of the cavity as described in previous embodiments, the flat surface inhibits rotation of the housing 467 along the y axis as shown in
In accordance with one implementation, the device 466 as configured in
An alternative arm or link 500, according to another embodiment, is depicted in
The lead screw 502 and nut 504 in this embodiment provide linear translation. More specifically, the motor 506 operates to turn the lead screw 502, which causes the nut 504 to move in a linear fashion. The string 508 attached to the nut 504 moves as a result, and this causes the joint component 512 to rotate, resulting in movement of the link 500 with respect to the link (not shown) connected at the joint component 512 (thereby changing the elbow angle at the joint).
The link 500 also has a compression or tension spring 514 positioned between the two cover components 516, 518 positioned to at least partially cover the motor 506. The spring 514 operates to maintain string 508 tension by urging the two components 516, 518 outward away from each other. Further, during the use, the spring 514 provides some passive compliance by allowing for relaxing the tension on the string 508 as the link 500 and other links of the operational component of the device are bent or twisted, such as during insertion into the patient's body. The relaxing of the tension allows for the links to move with respect to each other, thereby allowing for some bending and twisting of the device and thus making insertion somewhat easier.
In accordance with one embodiment, a magnetically coupleable robotic device system can include an insertion component that is used to insert the robotic device into the patient's stomach during a natural orifice procedure as described in further detail below. In one aspect, the insertion component is a sterile tubular component (also referred to herein as an “insertion overtube”). In one embodiment, in which the device is inserted into the body using a standard upper endoscope, the overtube is sized for both the robotic device and the endoscope.
Any of the magnetically coupleable robotic device embodiments described above can have a light component. For example, the light component in one embodiment is a light component 370 similar to that depicted in
The imaging component used with any magnetically coupleable robotic device can be a camera or any other imaging device. In certain embodiments, the imaging component can be any imaging component as described above with respect to mobile robotic devices or the fixed base robotic devices. Regardless, the camera can be any known imaging component that can be used with any of the magnetically coupleable robotic devices contemplated herein. In one embodiment, the imaging component is a stereo camera that creates a three-dimensional image.
The imaging component can help to increase or improve the view of the area of interest (such as, for example, the area where a procedure will be performed) for the user. According to one embodiment, the imaging component provides real-time video to the user. Alternatively, the imaging component can be any imaging component as described above with respect to the mobile robotic devices or the fixed base robotic devices.
In one aspect, the at least one actuation component described herein with respect to the magnetically coupleable robotic devices can be permanent magnet DC motors, shape memory alloys, piezoelectric-based actuators, pneumatic motors, hydraulic motors, or the like. Alternatively, the drive component can be any drive component as described in detail above with respect to mobile robotic devices or fixed base robotic devices.
Various embodiments of the magnetically coupleable robotic devices have an adjustable-focus component, some of which are described above. A variety of adjustable-focus components or mechanisms are known in the art and suitable for active or passive actuation of focusing an imaging component. Alternatively, the adjustable focus component can be any such focus component as described in detail above with respect to mobile robotic devices or fixed base robotic devices.
According to one aspect, any magnetically coupleable robotic device embodiment described herein is connected to an external controller via a connection component. In one embodiment, the connection component is a wired connection component that is a seven conductor cable that is configured to carry two video signals, electrical power, and operational signals from the controller. In this embodiment, the device can also have a microprocessor to decode any incoming operational signals and provide commands the device components. For example, the microprocessor can be an 8-bit embedded microprocessor (such as, for example, an 80C5X2 Core, available from Atmel Corporation located in San Jose, Calif.) with a full speed on-board USB interface. The interface receives input commands from the controller and the processor has 34 digital I/O pins to interact with component circuitry, such as motor drivers, focus mechanism, camera settings, etc. Alternatively, the microprocessor can be any known microprocessor that can be used for any robotic device as described herein.
Alternatively, the connection component is any wired or flexible connection component embodiment or configuration as described above with respect to mobile or fixed base robotic devices. In a further alternative, the connection component is a wireless connection component according to any embodiment or configuration as described above with respect to mobile or fixed base robotic devices. The receiver and transmitter used with a wireless robotic device as described herein can be any known receiver and transmitter, as also described above. According to another implementation described in additional detail above with respect to the mobile and fixed base devices, any magnetically coupleable robotic device embodiment described herein can be connected via a (wired or wireless) connection component not only to the external controller, but also to one or more other robotic devices of any type or configuration, such devices being either as described herein or otherwise known in the art.
In one embodiment, the data or information transmitted to the magnetically coupleable robotic device could include user command signals for controlling the device, such as signals to move or otherwise operate various components. According to one implementation, the data or information transmitted from the robotic device to an external component/unit could include data from the imaging component or any sensors. Alternatively, the data or information transmitted between the device and any external component/unit can be any data or information that may be useful in the operation of the device.
In accordance with one implementation, any magnetically coupleable robotic device as described herein can have an external control component according to any embodiment as described above with respect to the mobile or fixed base robotic devices. That is, at least some of the magnetically coupleable devices herein are operated not only by an external magnet, but also by a controller that is positioned at a location external to the animal or human. In one embodiment, the external control component transmits and/or receives data. In one example, the unit is a controller unit configured to control the operation of the robotic device by transmitting data such as electronic operational instructions via the connection component, wherein the connection component can be a wired or physical component or a wireless component. Alternatively, the external unit is any component, device, or unit that can be used to transmit or receive data.
In one embodiment, in which the magnetically coupleable robotic device has arms and joints similar to those embodiments depicted in
In use, the controller can be used to control the movement or operation of any components of the device such as the camera component, a sensor component, or any other component. For example, one embodiment of the controller controls the focus adjustment of the camera, and further controls the panning and/or tilting functions of the device.
According to one embodiment, the control component is configured to control the operation of the imaging component, the panning component, and the tilting component of a robotic device such as the device 380 depicted in
In accordance with one embodiment, the control component also serves as a power source for the robotic device.
According to one implementation, the magnetically coupleable robotic device is coupled to an image display component. In one embodiment, the image display component is a component of the controller. In one embodiment, the image display component is a commercially-available stereoscopic 3-D image display system. Such systems use images from two video sensors and display the images in such a way as to create a 3-D effect. For example, the image display component can be a Sharp LL-151-3D computer monitor. Alternatively, the image display component is special wireless eyewear that rapidly switches between images from the two sensors, such as, for example, the CrystalEyes 3™, which is available from Real D, located in Beverly Hills, Calif. Alternatively, the image display component can be any image display component as described above with respect to the mobile or fixed base robotic devices.
A magnetically coupleable robotic device as described herein, according to one implementation, has a power source or power supply. According to one embodiment, the power source is any power source having any configuration as described above with respect to the mobile or fixed base robotic devices. According to various embodiments, power can be provided by an external tether or an internal power source. When the device is wireless (that is, the connection component is wireless), an internal power supply can be used. Various implementations of the magnetically coupleable robotic devices can use alkaline, lithium, nickel-cadmium, or any other type of battery known in the art. Alternatively, the power source can be magnetic induction, piezoelectrics, fluid dynamics, solar power, or any other known power source. In a further alternative, the power source is a power unit positioned within the patient's body. In this embodiment, the power unit can be used to supply power not only to one or more robotic camera devices, but can also to any other surgical robotic devices.
In one embodiment, the magnetically coupleable robotic device has one or more sensor components. In various embodiments, such sensor components include any of the sensor components as described above with respect to the mobile or fixed base robotic devices.
According to one embodiment, any of the components on any magnetically coupleable robotic device as described herein can be known, commercially available components.
Although the above embodiments have included magnetic coupling components, it is understood that other attachment components or devices can be used to removably attach any of the device embodiments disclosed above or throughout the specification to an interior portion of a patient. For example, the attachment component could be a clip, a pin, a clamp, or any other component that provides for attachment or positioning along an interior surface of a patient.
Further, It is understood that any of the components disclosed herein with respect to any particular embodiment of a robotic device are also intended to be capable of being incorporated into any other robotic device embodiment disclosed herein. For example, any component disclosed with respect to a magnetically coupleable robotic device embodiment can also be incorporated into any embodiment of a mobile or fixed base robotic device as described herein.
Methods of Using Robotic Devices
Any of the robotic devices described herein can be used in various different surgical methods or procedures in which the device is used inside the patient's body. That is, the robotic devices can be used inside the patient's body to perform a surgical task or procedure and/or provide visual feedback to the user.
According to one embodiment, any of the mobile devices described above can be inserted entirely into the patient, wherein the patient can be any animal, including a human. In known laparoscopic procedures, the use of small incisions reduces patient trauma, but also limits the surgeon's ability to view and touch directly the surgical environment, resulting in poor sensory feedback, limited imaging, and limited mobility and dexterity. In contrast, the methods described herein using the various robotic devices inside the body can provide vision and surgical assistance and/or perform surgical procedures while the robotic device is not constrained by the entry incision.
In one embodiment, any of the above devices can be used inside an abdominal cavity in minimally invasive surgery, such as laparoscopy. Certain of the devices are sized and configured to fit through standard laparoscopic tools. According to one embodiment, the use of a robotic device inserted through one standard laparoscopy port eliminates the need for the second port required in standard laparoscopic procedures.
According to one embodiment, robotic devices as described herein having a camera can allow for planning of trocar insertion and tool placement, as well as for providing additional visual cues that will help the operator to explore and understand the surgical environment more easily and completely. Known laparoscopes use rigid, single view cameras with limited fields of view inserted through a small incision. To obtain a new perspective using this prior art device often requires the removal and reinsertion of the camera through another incision, thereby increasing patient risk. In contrast, the robotic devices with cameras as described herein provide one or more robots inside an abdominal cavity to deliver additional cavity images and easy adjustment of the field of view that improve the surgeon's geometric understanding of the surgical area. The ability to reposition a camera rapidly to arbitrary locations will help the surgeon maintain optimal orientation with respect to other tools.
In accordance with one implementation, any of the mobile robotic devices described herein can be used not only in traditional surgical environments such as hospitals, but also in forward environments such as battlefield situations.
According to another embodiment, any of the robotic devices described herein can be used in a natural orifice procedure. “Natural orifice surgery,” as used herein, is any procedure in which the target portion of the body is accessed through a natural orifice such as the mouth, anus, vagina, urethra, ear, or nostril, or any other natural orifice, for surgical or exploratory purposes.
For purposes of this application, the umbilicus is deemed to be a natural orifice. More specifically, the umbilicus is a natural orifice that can be reopened for use in a surgical or exploratory procedure and then subsequently allowed to heal closed again.
Natural orifice surgery, according to one embodiment, can be performed by inserting an appropriate medical device into the body through the mouth and penetrating into the abdominal cavity via an incision in the stomach wall, which is also referred to as “transgastric” surgery. In one embodiment, the gastrotomy (a hole in the stomach wall) is formed using a standard endoscopic tool. Alternatively, the gastrotomy is formed using one of the robotic devices.
One advantage of such surgery is the elimination of skin incisions and a reduction in post-operative pain and/or discomfort. Another advantage of natural orifice surgery through the gastric cavity is the substantially antiseptic state of the stomach, thereby reducing the risk of infection. Another advantage is the rapid healing characteristics of the stomach. That is, gastric incisions heal more quickly than incisions made in the abdominal wall. Natural orifice surgery eliminates skin incisions and reduces post-operative pain and discomfort. Such an approach provides a distinct benefit compared to conventional laparoscopy where multiple entry incisions are required for tools and a camera. Thus, access through a natural orifice eliminates the need for external incisions, thereby avoiding possible wound infections while reducing pain, improving cosmetics, speeding recovery, and reducing adhesions and ileus. Further, natural orifice procedures can also for the first time allow minimally invasive techniques to be used on obese patients for whom the thickness of the abdominal wall makes laparoscopy impossible.
In accordance with one aspect, the method of performing natural orifice surgery includes performing the procedure with an untethered robotic device. Alternatively, the method relates to a method of performing natural orifice surgery with a robotic device that is tethered with a flexible connection component. The device can be any of the robotic devices disclosed herein. Alternatively, the device can be any robotic device that can be inserted into a natural orifice of the body for surgical or exploratory purposes. In a further alternative, the device can have any known form or structure so long as the device is a robotic device that can be inserted into a natural orifice for surgical or exploratory purposes.
According to another embodiment, any one of the robotic devices disclosed herein can be used with one or more other robotic devices, including any of the devices disclosed herein. That is, the robotic devices disclosed herein constitute a family of robotic devices that can be utilized together and/or in combination with other known robotic devices to perform surgical procedures. That is, any combination of the robotic devices can be positioned inside the patient's body to cooperatively perform a surgical procedure.
In one implementation, the two or more robotic devices, whether coupled in an untethered fashion or via a wired connection component, can be operated in cooperative or sequential fashion or any other fashion during a procedure in which more than one robotic device provides an advantage. In another embodiment, multiple mobile, fixed-base, and/or magnetically coupleable devices with a variety of sensors and manipulators are used cooperatively as a low-cost robotic surgical “team” that are inserted into the patient's body through a single incision. This family can perform an entire procedure while being remotely controlled by the user.
One example of more than one robotic device being used cooperatively, according to one embodiment, is depicted in
Further, it is contemplated that multiple robotic camera devices can be used simultaneously to provide the operator with improved visual feedback from more than one viewing angle. Likewise, the one or more robotic camera devices can be used in conjunction with one or more surgical robots.
In a further embodiment, a process can be implemented during surgical procedures so that the number and location of all wireless robots can be documented throughout a procedure.
In accordance with one implementation, the cooperative method can be combined with the natural orifice method. That is, multiple robots, each with various different functions, could be inserted into the patient's body through a natural orifice. This method allows multiple robots to be independently inserted through the orifice, thereby providing a surgical “team” inside the patient's body during a surgical procedure. In one embodiment, the current method allows sufficient room in the esophagus to remove discarded tissue (such as a gall bladder) and for insertion of specialized tools (cauterizing, etc).
Another embodiment relates to methods, systems and devices for cooperative use of a robotic device with (1) standard laparoscopic tools, (2) the da Vinci® system, and/or (2) at least one other robotic device, including any of the devices discussed or referenced above, or any combination thereof.
In one embodiment, a robotic camera device can be used in conjunction with a standard laparoscope to give the surgeon an auxiliary viewpoint, such as, for example, a rear viewpoint of an abdominal feature. In another embodiment, the robotic camera device can be used by itself to reduce patient trauma by inserting it through a tool port. In another embodiment, the robotic camera device is used as the camera or cameras for a minimally invasive abdominal surgery where the camera or cameras can be moved to any position inside the cavity, eliminating the need for the laparoscope. This requires only two incisions in the abdominal wall instead of three, reducing patient trauma and risk of complications.
According to one embodiment, robotic devices disclosed herein cooperate with da Vinci® tools, thereby complimenting the da Vinci® system with auxiliary viewpoints and thus improving visual feedback to the surgeon. One or more of the robotic devices are placed entirely within the abdominal cavity and are therefore not constrained by the entry incisions.
In one example, two robotic devices can be used in cooperation with the da Vinci® system during a surgical procedure. The first device is a magnetically coupleable pan-and-tilt robotic camera device that is attached to the abdominal wall using magnets. The second is a wheeled mobile robotic device with a camera. The pan-and-tilt device provides a view from above the surgical target while the mobile device provides a view from a low perspective. The point-of-view of both these devices is easily changeable throughout the procedure. In one embodiment, the video from these devices is sent directly to the da Vinci® console and can, by the surgeon's choice, be displayed as one image in the stereo-vision system. In another embodiment, both devices are repositioned throughout the surgery to give perspectives that would otherwise require a new incision and a time consuming repositioning of da Vinci® tools. In one embodiment, the robotic devices are controlled by the surgeon via a separate joystick.
In one embodiment, the da Vinci® system is positioned as per normal procedure. Three small incisions are made in the abdominal wall for the two tool ports and the laparoscope. A special, slightly larger, trocar is used for insertion of the robotic devices that allows for the devices' electrical wire tethers. Alternatively, the robotic devices are wireless. The remaining trocars are then placed and the abdomen is insufflated. The da Vinci® tools and laparoscope are then inserted and readied for the surgery. The robotic devices are then powered and the pan/tilt device is lifted from the organs to the upper surface of the abdominal wall using a magnet holder outside the abdomen. The robotic devices can be positioned using their cameras, the da Vinci® tools, or the laparoscope. Once the robotic devices are properly positioned, the da Vinci® video input is switched from the standard laparoscope to the hanging device. The robotic devices' functions are then checked to establish proper operation and lighting. The operating surgeon then begins the procedure. In one embodiment, the robotic devices can be repositioned and the pan/tilt features can be actuated to track tool movements during the procedure. The procedure can then be performed using the da Vinci® system tools but with primary video feedback coming from the devices. After the procedure, the robotic devices are moved back to the special trocar, the abdomen is deflated, and the robotic devices are retracted.
Those skilled in the art will understand that the process described represents merely one embodiment and that the order described could be varied and various steps could be inserted or removed from the process described.
The process described above and similar procedures show the benefits of using robotic devices to assist surgeons by cooperative use of more than one cooperative device, including in certain embodiments using at least one robotic device cooperatively with the da Vinci® system. In this embodiment, the robotic devices provide complimentary visual feedback to the surgeon during a procedure. The multiple viewpoints improve the understanding of the surgical environment, thus demonstrating how at least one robotic device can cooperate with each other or with the da Vinci® system to improve surgical care.
In one embodiment, unobstructed access to the surgical site is achieved by a device designed to allow for mobility and flexibility in placement while being configured for use in the already limited space of the abdominal cavity. In the present embodiment, a cooperative surgical environment is achieved by suspending a robotic device from the abdominal wall in a fashion that allows for mobility in placement within the abdominal cavity. Functionality through useful video feedback of the appropriate surgical site is also provided. In another embodiment, the device can pan and tilt the camera as well as focus on objects at differing distances within the abdominal cavity.
In another embodiment, a hanging pan/tilt robotic device is used cooperatively with the da Vinci® system to perform a surgical procedure. The hanging device provides the primary (non-stereo) visual feedback to the da Vinci® console. It is repositioned and actuated throughout the procedure to optimize the feedback available to the surgeon.
In another embodiment, video feedback to the da Vinci® console from the robotic device is provided to only one of the console's two eyepieces. The surgeon controls the pan/tilt device functions from the console via a separate joystick. The multiple viewpoints available through the use of the cooperative robotic device improves understanding of the surgical environment.
In another embodiment, a da Vinci® procedure utilizing device visual feedback demonstrates the implementation of cooperative devices in minimally invasive surgery. The additional feedback is invaluable and allows the surgeon to scan the surgical site from varying angles. The pan/tilt device suspension system also allows for repositioning of the device throughout the procedure without necessitating multiple incisions for the laparoscopic arm.
In one embodiment, a natural orifice procedure can include an insertion component that is used to insert the robotic device into the patient's stomach. In one aspect, the insertion component is a sterile tubular component (also referred to herein as an “insertion overtube”). In one embodiment, in which the device is inserted into the body using a standard upper endoscope, the overtube is sized for both the robotic device and the endoscope.
One method of natural orifice procedure, according to one embodiment, includes advancing a sterile overtube into the patient's stomach with a standard upper endoscope and irrigating the stomach with antibiotic solution. The robotic device is then inserted into the gastric cavity through the overtube. The robot is then inserted into the abdominal cavity through a transgastric incision created with an endoscopic needle-knife. The incision can be approximately the same diameter as the robot. Finally, the device is retracted into the gastric cavity. Subsequently, endoscopic closure of the transgastric incision can be accomplished using two endoclips and one endoloop. Further, the robotic device is grasped with an endoloop and retracted back through the esophagus.
Although the present invention has been described with reference to preferred embodiments, persons skilled in the art will recognize that changes may be made in form and detail without departing from the spirit and scope of the invention.
One factor to consider in the development of the mobile robotic devices was the amount of torque needed to move the device.
To calculate the needed torque, a free-body diagram of the robot sitting motionless on a slope was used to calculate the torque required to keep the robot stationary on the slope. This calculation would be the stall torque that the motor would need (provided that the friction of the surface was enough to prevent the wheels from slipping). The free-body diagram is shown below in
From this free-body diagram the following equations were written:
(W sin θ)r=(ma)+/α+τ
W sin θ−f=ma
W cos θ=N
This results in the following:
τ=(W sin θ)r
where
The robot was modeled as a solid aluminum cylinder 15 mm in diameter and 76 mm long. A solid aluminum cylinder of this size would have a mass of 36.4 g and a moment of inertia of 1.02 [kg-m2]. The resulting calculations show that for the robot to hold its position on a slope of θ degrees a torque, τ, is needed (Table 1).
After determining what torque was required to move the robot, a motor and a gearhead were selected that would reduce the speed and increase the torque output from the motor. Two motors were tested to determine if they met the torque requirements. The first motor was a standard, commercially-available 6 mm diameter pager motor and the second was a 6 mm blue motor taken from a toy ZipZap™ remote-controlled car, which is available from Radio Shack.
Tests determined the stall torque of each motor per volt input. For the test, a bar was placed on the motor shaft and a voltage was applied to the motor. The angle at which the bar stalled was then measured for each applied voltage. The torque that was present on the motor shaft was calculated and plotted versus the voltage, and a linear fit was used to determine the stall torque/volt of the motor. The results of the test are shown in Table 2.
The results of this test show that neither motor supply enough torque to hold the mobile robot on more than a minimal slope. The ZipZap™ motor can provide 0.057 [mNm] at 3 V and the pager motor can supply 0.084 [mNm] at 3 V. Both motors could only hold the robot stationary on a 15 degree slope.
Another motor tested was model SBLO4-0829 with gearhead PG04-337, available from Namiki. The motor runs on 3 V and testing determined that it can provide 10.6 [mNm] stall torque at 80 rpm. This motor provides a design factor of 4 for the robot on a 75-degree slope (if frictional force is sufficient to prevent sliding).
Wheel Friction
The friction characteristics of two wheels were tested.
The device tested was a robot having a weight (“W”) of 1.0 oz. The radius of the two wheels was 7.5 mm, and they were made of aluminum.
Experiments were conducted on top of four types of objects: a tabletop, a mouse pad, particleboard and sliced beef liver. The robot was placed on top of each of these objects and the maximum friction force, F, was measured. The force was measured using an Ohaus Spring Scale with one-quarter ounce divisions. The force was approximated to the nearest 0.05 ounces.
The coefficient of friction was determined by the formula μ=F/W. Table 3 shows the four coefficients of friction measured by experiments.
Additional force analysis was also applied to the two-wheeled device described above. That is, the amount of required frictional force was determined in the following manner.
The force analysis was based on an elastic foundation, i.e., where the mobile robot was assumed to roll on an elastic surface (see
Then from the equation above,
Thus, the sum of partial differential friction force:
Σf=δ(θ)cos(θ)+τ(θ)sin(I)
By the integral calculation, one can get the friction force:
here Σ is the Young's modulus and R is the Poisson's ratio.
From the force analysis, it was determined that the frictional force was proportional to the weight and inversely proportional to the radius of the wheel. Therefore, either of the following two methods could be used to influence frictional force. First, the mass of the robot could be increased. One good way to do so would be to change the material of the wheels. Second, the radius of the wheels might be reduced. Another solution is to add treads to the wheels. Alternatively, the tips of the treads may have a smaller radius without reducing the diameter of the wheel itself.
In this example, a velocity analysis was performed on a manipulator arm for a mobile robot, according to one embodiment discussed above.
When performing such an analysis, it was helpful to define a matrix quantity called the Jacobian. The Jacobian specifies a mapping from velocities in joint space to velocities in Cartesian space. The Jacobian can be found for any frame and it can be used to find the joint torques, discussed infra.
The fundamental equations used in finding the Jacobian are:
For link 1,
For link 2,
For link 3, i=2
For link 4, i=3
where Sn sin θncn=cos θn, snm=sin(θn+θm), cnm=cos(θn+θm).
The second method provides the results seen in
The motor selected for the manipulator in this example was a 6 V DC Micromotor manufactured by Faulhaber Company. The 6 V motor had a 15,800 rpm no-load speed, 0.057 oz-in stall torque, and weighed 0.12 oz. The motor had an 8 mm diameter and it was 16 mm long.
Due to its high no-load speed, a precision gearhead was used. The precision gearhead used was a planetary gearhead. For the preliminary analysis, a gearhead with a reduction ratio of 256:1 was selected. It had an 8 mm diameter, is 17.7 mm long, and weighs 0.19 oz.
A 10 mm magnetic encoder was chosen for this particular examination. It was 16.5 mm long, but it only added 11.5 mm to the total length of the assembly. The weight of the encoder was assumed to be 0.1 oz. The encoder provided two channels (A and B) with a 90° phase shift, which are provided by solid-state Hall sensors and a low inertia magnetic disc. Table 4 shows a summary of motor, planetary gearhead, and encoder properties.
For purposes of the following calculations, it was assumed that the links were cylindrical tubes, as shown in
Link Volume:
Link Mass:
Total Weight of Motor and Link:
m=mτ+mL
m=11.6233 g+1.49152 g=13.1148 g
m1=m2=m
Payload Mass:
mp=5 g
Moment Calculations (Refer to
Since L1=L2=L
It was calculated based on the above equations that the maximum torque allowed by the motor for a continuous operation is 8.5 oz-in, which is 0.41 mNm. Using the reduction ratio of 256:1, the maximum torque allowed is 104.86 mNm (256×0.41 mNm).
As discussed above, precision gears with other reduction ratios may also be used, according to various embodiments. Tables with calculations for lower reduction ratios are provided below. These calculations are exemplary and are not intended to be limiting in any fashion.
By using the Jacobian that was previously developed and is shown below, it is possible to calculate the torques provided by the force exerted to the tip of the manipulator used in this example. However, it should be noted that this method does not take into account the weights of links and motors.
where
and L=59.5 mm
Using θ1=0°, θ2=90°, θ3=0°
Thus the torque for the base motor is 0 mNm: for link 1 it is 5.836 mNm, and for link 2 it is 2.918 mNm. This result makes sense because the largest torque will be exerted on the joint farthest away from the tip of the manipulator. Also, since the distance is two times the distance to middle joint, the result is two times bigger.
Accounting for the link and motor masses,
The total torque is,
As shown, both methods provide the same result.
In the embodiment of the manipulator arm robot used in this example, the electronics and control consisted of four major sections described above in the detailed description and depicted in block diagram form in
The first section of the hardware in this embodiment was a PC with Motion Engineering, Inc. PCI/DSP motion controller card. This card used an Analog Devices DSP chip running at 20 MHz to provide closed-loop PID control of up to four axes simultaneously. It had encoder inputs for positional feedback. The servo analog outputs were controlled by a 16-bit DAC, which allowed very precise output control. The card also featured several dedicated digital I/O functions, including amplifier enable output, amplifier fault input, home input, positive limit input, and negative limit input. However, only the basic functions were used in this application: servo analog output and digital encoder inputs. The PCI/DSP came with a full-featured C programming library to aid in programming different motion functions. Also provided was a Windows-based program, Motion Control, to configure and tune the controller, as well as to capture data from simple one-axis motion profiles.
The output from the PCI/DSP was an analog signal with a range of ±10V. In order to interface with the microcontroller, this signal was converted to a 0.5V range. Two simple op-amp circuits performed this function. Both op-amp circuits used the LM318 op-amp from National Semiconductor. The first section was a standard inverting circuit with a gain of −0.25. This converts the ±10V input into a ±2.5V output. This circuit is shown in
With V2 a constant 2.5V, an output voltage of 0-5V results. This circuit is shown in
Capacitors were placed at the output of each op-amp to filter out high frequency noise. This two-amplifier circuit is duplicated exactly for each axis. The 2.5V reference is supplied by a 10 K potentiometer.
After the analog voltages were scaled and shifted, each was sampled by the PsoC (Programmable System on a Chip) microcontroller and converted to a PWM output signal and a direction signal. The PsoC also provides direction output based on the input voltage. The PsoC is made by Cypress Semiconductor, and is an 8-bit microcontroller with several generic digital and analog “blocks” that can be configured using the PsoC Designer software package to perform many different functions. These functions include, but are not limited to: ADCs, DACs, PWM generators, timers, UARTS, LCD drivers, filters, and programmable amplifiers. PsoC Designer also provides an API accessible from C and assembly to interface with these on-board components. For the embodiment described here, a single ADC, an analog multiplexer, and three PWM generators were used. The duty cycle of the PWM outputs are directly proportional to the analog input signals. Table 8 summarizes the function of the microcontroller.
The outputs of the microcontroller circuit were fed to the inputs of the FAN8200. These were H-Bridge Driver circuits, in a 20-pin surface mount package. Each driver had an enable and direction input. For this embodiment, the PWM signal was fed to the enable input, and the direction output of the microcontroller was fed to the direction input of the motor driver. The motors on the robot were connected directly to the PCI/DSP card, with no signal conditioning required. As mentioned previously, the PsoC microcontroller sampled each of the three analog outputs, and updated the corresponding PWM duty cycle and direction output accordingly.
The majority of the code was executed in the ADC interrupt service routine. A flowchart of the ISR is shown in
The other software element in the system was the PC program that was used for testing the robot. This was a console-based Windows program that used the Motion Engineering library to send commands to the PCI/DSP. This program can move each axis individually, or move all three simultaneously using the DSP's coordinated motion functions, allowing the user to enter a desired position, in encoder counts, for each axis. The DSP card then creates an appropriate motion profile, and moves each motor to the correct position. This program also was used to generate impulse responses for each motor for analysis.
There are several techniques available for designing system controls; here, modern control theory was used for control design of a three link robotic arm. A typical modern control system contains a plant and a controller in the feed forward. This design theory is shown in
In this case, modern control theory was used to design three separate controllers. Three controllers were required in order to control the three motors used to manipulate the arm. In order to do this, it was assumed that three separate systems exist. Each system was designed assuming that only one motor, the motor being controlled in the system, was active. This was acceptable based on the method for determining the reaction of a system to a disturbance.
Shown in
In one embodiment, there are several methods a designer can use to design a plant. Most methods used are analytical. In this case an experimental approximation of the plant was created. This was an effective and verifiable method for approximating the system. To collect the experimental data, a computer program was used to send a voltage impulse to the motor. The program simultaneously recorded the position of the motor, using the encoder. This procedure was performed three separate times, once for each motor. The data was then used to construct plots of motor position (based on encoder counts) versus time in seconds. Plots from the data are shown in
From analyzing the data in
Multiple configurations of the plant were used to find the approximation to the experimental plant. This included the use of complex poles, as well as changing the number of poles and zeros in the transfer function. From these configurations, it was determined that the plant, G(s), can be modeled using the transfer function in the continuous time domain shown the following in equation. In this equation, the poles are 0, −b and −c, and the zero is −α.
Using the simplex search program, along with the error subroutine, the following system plant values were determined:
System for axis 1:
System for axis 2:
System for axis 3:
Since all motors were identical, they should have similar system poles and zeros, even though they are located in different positions on the robot. This was shown to be true for the systems for axis 1 and 3. However, the system for axis 2 did not conform to the other two systems very closely. This was most likely due to poor data. A larger impulse on the motor for axis 2 would have helped to obtain more realistic data.
To see how well the system in the continuous time domain reflected the data taken from the digital system, the error subroutine was used once again. This time the error subroutine was compiled as a program rather than as a subroutine. By substituting the above values for a, b and c into the error program, the continuous fit was mapped to the actual digital data. The results were plotted once again as motor position (based on encoder counts) versus time in seconds. These plots are shown in
To control the motor positions on the robot, a PID controller was used. When using a PID controller, the controller from
Where Kp is the proportional constant, KD is the derivative constant, and KI is the integral constant. With the PID controller, the system becomes a type 2 system. This means that the error in the response to a step and ramp input is zero. However, the error for the response to a parabolic input is 1/Ka. Where Ka is the acceleration constant and is defined as:
Since the input can be defined, a parabolic input is not used.
Computing the values for Kp, KD and KI was done using Routh Analysis along with Ziegler-Nichols tuning. Routh Analysis uses the characteristic equation of the system transfer function. In this case, though, D(s)=Kp, only. The transfer function of this system with gain only, using G(s) as defined above, is shown in the following equation.
Note that Routh Analysis only can be used if the system for D(s)=1 is stable. This is true if the characteristic equation of the system when D(s)=1 has stable roots. Stable system poles, or roots of the characteristic equation, are roots that have negative real values or are located at the origin. The following equation is the characteristic equation for the system when D(s)=1.
CE=s(s+b)(s+c)+(s+a)
The following poles or roots of CE are:
System for axis 1:
System for axis 2:
System for axis 3:
Since all poles have negative real parts, the uncontrolled system was stable and Routh Analysis can be used.
Using the characteristic equation, or the denominator from the equation, solving for TF, above, Routh Analysis is performed as follows:
Using Maple V, the term (b1*s) is set equal to zero and then solved for Kp=Kp(max). The results are as follows:
System for axis 1:
System for axis 2:
System for axis 3:
These results were all obtained using Maple V.
In order to use Ziegler-Nichols tuning with Routh Analysis, the system period was also needed. The system period was found by setting s=jω, Kp=Kp(max). and solving for w (system frequency in rad/s) from the following equation.
α1(jω)2+α3=0
Since,
ω=2πf
Then the system period in seconds was:
The resulting system periods were as follows:
System for axis 1:
System for axis 2:
System for axis 3:
With the Ziegler-Nichols tuning equations for Kp, KI, and KD, the controller, D(s), as defined above, was designed. The Ziegler-Nichols tuning equations for PID control are shown below.
The resulting values for Kp, KI, and KD are as follows:
System for axis 1:
System for axis 2:
System for axis 3:
The resulting system with PID control for all systems is shown in
One way to decide if these PID values were reasonable was to do a root locus plot of the open loop transfer function, D(s)*G(s). System stability also could be found from the root locus plot. That is, the poles or roots of the characteristic equation on the root locus should be located in the negative real plane. These plots, shown in
As shown in
All responses shown in
Adjusting the phase margin of a system through the use of a lead or a lead-lag compensator is a technique that generally reduces the percent overshoot of a system. The phase margin is the angle between the negative abscissa and the point on the Nyquist diagram of the system, where the magnitude is 1. In most cases, a phase margin of about 60 degrees is optimal for reducing percent overshoot.
From using a Nyquist plot program, the following data was obtained.
System for axis 1:
To compensate for phase loss due to the lag compensator:
System for axis 3:
To compensate for phase loss due to the lag compensator:
There are a few things to note. Once again, the data for axis 2 resulted in compensator design for axes 1 and 3 only. Also, ωc may be changed to any desired frequency. G(jω), and φ(added) would subsequently change depending on the phase and magnitude at the selected ωc. However, the phase margin would remain the same.
The following equations were used to define a lead and lag compensator, respectively.
The resulting compensators from equations 11 and 12 for systems for axes 1 and 3 were as follows:
Compensator for axis 1:
Compensator for axis 3:
The lead and lag compensators are integrated into the design as shown in
Since zeros placed closer to the origin than poles create overshoot, the lead compensator was placed in the feedback. This is because if placed in the feed forward, a zero would be located between the origin and a pole in the root locus plot. For this same reason, the lag compensator was placed in the feed forward.
The effect of these compensators on the system was analyzed. First, the Nyquist plot program, was used once again. This was done to see what effect the compensators had on the phase margin. Finally, a plot of the response of the systems to a unit step input was made using the Maple V program 1.
Resulting data from the Nyquist plot program:
System for axis 1:
System for axis 3:
This was proof that the compensator design was successful in adjusting the phase margin to the desired 60 degrees of phase. Shown in
As shown in
Once the controller design was complete in the continuous time domain, it could be converted to the discrete time domain. This is required in order to control a digital system. However, it was only necessary to convert the compensators and controller to the discrete time domain. When this was done, a control algorithm was introduced to the computer program.
To convert the compensators and controllers to the discrete time domain or z-domain, Tustin's method was used. Tustin's method is only good for linear systems and introduces the relationship shown in the following equation.
where T represents the sampling period of the controller. Substituting this equation into the controller, lead compensator, and lag compensator yields the following equations.
The final system block diagram of this embodiment is shown in
In
After the designed components were assembled, a test was performed to verify the controllability and accuracy of the manipulator used in this example. The tip of the manipulator, which was attached to a camera, is supposed to move through four points along the sides of the triangle shown
To test the accuracy of the movement of the tip, the assumed motor rotation angles were input into the controlling program. These input angles controlled the tip movement along the edges of the triangle. Table 9 shows the motor rotation angles, in encoder counts, for four different points. The ratio of encoder counts per degree was 28.9.
The next step was to use the Jacobian to transfer the encoder counts to the xyz coordinates:
L1=83 mm, L2=L3=59.5 mm, and t1, t2, t3 represent the motor angles in encoder counts of axes 1, 2 and 3.
Shown below in Table 10 are the results of x, y and z coordinates for the four different points.
The distance between the four points was then calculated by using the equation shown:
Dist=√{square root over ((x1−x2)2+(y1−y2)2+(z1−z2)2)}{square root over ((x1−x2)2+(y1−y2)2+(z1−z2)2)}{square root over ((x1−x2)2+(y1−y2)2+(z1−z2)2)}
The actual encoder reading was found to describe the movement of the manipulator tip. Shown below in Table 11 are the distances between the four points.
The difference between the measured displacement and calculated displacement indicates there is a big error between the two. This was due to several error sources, in the measurement of link lengths L1, L2 and L3, and due to the estimated ratio of the encoder counts to degrees. A source of mechanical error is backlash at the gear mesh.
The goal of the current study is to demonstrate the capability of introducing a mobile robot into the abdominal cavity through the esophageal opening.
In this study we used the mobile robotic device depicted in
An anesthetized pig was used as the animal model. The 60 lb. pig was fed Gatorade and water for 36 hours prior to the procedure. A sterile overtube was advanced into the pig's stomach with a standard upper endoscope. The stomach was irrigated with antibiotic solution.
The robot was inserted into the gastric cavity through the overtube. The robot explored the gastric cavity as shown in
Results
After insertion into the gastric cavity, the mobile robot successfully maneuvered throughout the cavity under EGD control (using visual feedback from the endoscope) (see
The mobile robot was capable of traversing the entire abdominal cavity, including the liver (see
After successfully exploring the abdominal cavity, the mobile robot was retracted into the gastric cavity. Closing the gastrotomy was successfully accomplished using endoclips and one endoloop. Retrieval of the miniature robot was accomplished without difficulty with an Endoscopic snare.
The ability to perform abdominal surgery without skin incisions can reduce patient trauma. However, the difficulties lie in performing these procedures using only EGD video feedback, and introducing sufficiently capable tools into the abdominal cavity. The ability to provide transgastric robotic assistance inside the abdominal cavity may help solve some of these problems. As the robot is not restricted by the length or the angle of the endoscope insertion it will by definition have a greater number of degrees of freedom. The working channel of the endoscope also limits the size and type of instrumentation available to the surgeon. Thus, a miniature robot could perform various surgical procedures and/or be used in conjunction with an endoscope or other surgical devices to achieve better visualization and greater mobility in the peritoneal cavity. According to one embodiment, the endoluminal robots of the present invention can be equipped with cameras and manipulators. The robots can provide surgical assistance. Further, a family of robots can working together inside the gastric and abdominal cavities after insertion through the esophagus. Such technology will help reduce patient trauma while providing surgical flexibility.
In the instant example, the effectiveness of using mobile camera robots to provide sole visual feedback for abdominal exploration and cholecystectomy was examined.
Methods and Materials
A mobile robotic camera device similar to the device depicted in
The device was inserted through a fabricated trocar into an anesthetized pig, and the abdominal cavity was then insufflated with carbon dioxide. The trocar was designed to accommodate the 20 mm diameter of the device. According to an alternative embodiment, the device will use standard 15 mm laparoscopic trocars. Next, a standard trocar was inserted to provide an additional tool port. A third port was also created to accommodate a standard laparoscope. The laparoscope was used to provide lighting for the camera of the mobile robotic device, but the surgeon did not use visual feedback from the laparoscope during the procedure.
Results
The surgical team used the device to help plan and view the additional trocar insertions and laparoscopic tool placements, as shown in
The device was also used to explore the abdominal cavity, as shown in
Finally, a cholecystectomy was performed with the device providing the only visual feedback available to the surgeon (i.e. the video from the laparoscope was not viewed by the surgeon), as shown in
Discussion
This successful experiment demonstrated that it is possible to perform a common laparoscopic procedure using an in vivo camera system as the sole source of visual feedback. This has the potential to reduce patient trauma by eliminating the need for a camera port and instead inserting mobile in vivo camera robots, such as the device used in this example, through one of the tool ports.
This example is an examination biopsy tool design for a mobile robotic device. The device should produce sufficient clamping and drawbar forces to biopsy porcine tissue.
To examine clamping and drawbar forces used during a biopsy, experimental biopsies were conducted. A biopsy forceps device that is commonly used for tissue sampling during esophago-gastroduodenoscopy (EGD) and colonoscopies was modified to measure cutting forces during tissue biopsy. These forceps 560, shown schematically in
The diameter of the forceps (h) depicted in
For a cable force of 10 N, the force at the tip was approximately 1.4 N for this design where a was 2.9 mm, b was 1.7 mm, and d was 0.65 mm. The maximum area of the forceps in contact with tissue during a biopsy was 0.3756 mm2.
Assuming an even distribution of force, the applied pressure was approximately 3.75 MPa. However, by taking a smaller “bite”, the contact area was reduced and the pressure can be drastically increased and the required force was decreased.
A normal biopsy device was modified to contain a load cell 582 to measure clamping forces indirectly, as shown in
Measurements of cable force were made while sampling liver, omentum, small bowel and the abdominal wall of an anesthetized pig. Representative results for a liver biopsy are shown in
Generally, biopsy forceps do not completely sever the tissue. When this is the case, the forceps are gently pulled to free the sample. This extraction force also needs to be produced by a biopsy robot. The magnitude of the extraction force needed to be determined so that a robot could be designed to provide sufficient drawbar force to free the sample.
A laboratory test jig was built to measure the force needed to free a biopsy sample of bovine liver. After clamping the sample with the biopsy forceps, a load cell attached to the handle of the device was gently pulled to free the sample while the tensile force was recorded. Representative results shown in
As indicated, a complete cut of the tissue is rarely achieved and some tearing of the sample is needed to extract the sample. To obtain a biopsy sample, the in vivo robot embodiment of the present example should produce enough drawbar force to pull the sample free. A biopsy robot similar to the devices shown in
In the second test, for which results are depicted in
As depicted in
A direct current motor 600 drives the lead screw 602 vertically as the linkage 606 transforms the vertical motion of the lead nut 604 to the horizontal translation of the slider 608. This allows for a large mechanical advantage at the point when the graspers are nearly closed.
Force measurements were made in the laboratory to determine the maximum amount of force that could be produced using the biopsy robot embodiment of this example. Representative results from these tests are shown in
In vivo mobility testing with the embodiment discussed herein indicated that the wheel design of the instant embodiment produces sufficient drawbar forces to maneuver within the abdominal environment, allowing the robot to traverse all of the abdominal organs (liver, spleen, small and large bowel), as well as climb organs two to three times its height. These tests were performed without causing any visible tissue damage. Video recorded during one of the tests was used to reconstruct the path traversed by the robot, a portion of which is illustrated in
After exploring the abdominal environment, the biopsy mechanism described in this example was used to acquire three samples of hepatic tissue from the liver of the animal. The robot camera was used to find a suitable sample site. The biopsy graspers were opened and the sample site was penetrated with the biopsy forceps' spike. Then the graspers were actuated. This cut nearly all of tissue sample free. The robot was then driven slowly away from the sample site thereby pulling free the tissue sample. This tissue sample was then retrieved after robot extraction through the entry incision. This demonstrated the success of a one-port biopsy and successful tissue manipulation by an in vivo robot, according to one embodiment.
A laboratory two-component drug delivery system is shown in
The ability of this system to adequately mix liquid and solid components of a drug was evaluated in a series of bench top experiments. The liquid and solid drug components were simulated using commonly available materials (e.g., corn starch, dyed saline solution, etc). One visual metric of mixing efficiency is the color uniformity of the mixture as determined by measuring the RGB color components of the mixture using image processing software. Representative results are shown in
Bench top tests were also conducted to determine the force that could be applied by an actuation mechanism that could be incorporated into this type of drug delivery tool. One type of mechanism might use a permanent magnet direct current motor (MicroMo, 2005) with a lead screw mounted on the motor shaft. Rotation of the lead screw would move a lead nut attached to the fluid reservoir plunger in and out to dispense the two drug components. This concept was implemented in a test jig 180, illustrated in
Nagelschmidt (1999) found that the maximum force required to mix and dispense fibrin-based hemostatic agents through 1 mm diameter catheters 27 cm long was less than 5 N. These results strongly suggest that the actuation mechanism described above will generate sufficient forces to deliver dual component fibrin-based hemostatic agents.
This example presents a quantitative comparison of image quality between a robotic camera device according to one embodiment and a standard laparoscopic camera. Image analyses are presented for both the in vivo robot and a standard laparoscope, including an examination of the Modulation Transfer Function (MTF), color reproduction, and image distortion. Then the stereoscopic three dimensional reconstruction is analyzed in ex vivo experiments. Finally, the use of the in vivo stereoscopic robot is demonstrated during a cholecystectomy in an animal model. These results suggest that these in vivo devices can provide visualization of laparoscopic procedures that is comparable to standard laparoscopes and sufficient for laparoscopy.
The device tested in this example is depicted in
Single Camera Comparison
In this examination, the imaging device was a color digital CMOS image sensor from Micron. Further, the laparoscope used is a device with a Tricam™ SL NTSC control unit and a Xenon 175 light source, all manufactured by Karl Storz GmbH & Co. KG, located in Tuttlingen, Germany.
Visual metrics are often used to quantify quality differences between the large numbers of commonly available digital imaging devices. One such metric is the well established Modulation Transfer Function (MTF) used as a metric both for optical systems and digital imaging systems. This transfer function measures the amount of detail a given imaging system can display using a frequency domain measurement. The metric is usually expressed in units of spatial frequency, such as line pairs per mm (lp/mm) or cycles per pixel (c/p). The vision target used for MTF testing is an ISO 12233 Resolution chart printed on Kodak photo paper, measuring 196 mm×120 mm (7.75″×4.75″).
Color accuracy is another important image quality metric. One measurement of color accuracy is the use of a Macbeth color chart. The chart has 24 zones, 18 color and 6 grayscales. The target chart used for color error measurements is a Mini ColorChecker™. The ColorChecker™ is a standard Macbeth™ color chart, measuring 82 mm×57 mm (3.25″×2.25″).
Both these metrics as well as standard measures of distortion are used to quantify and compare the performance of the in vivo imaging robot. For distortion tests, a square grid was generated from the Imatest™ application, and printed using a laser printer. Imatest™ is a software package that can be used to evaluate different types of imaging systems.
All imaging tests (MTF, color error, distortion) were conducted with the same experimental setup. The setup held the imaging targets at a fixed distance and orientation with respect to the imager (in vivo camera and laparoscope). Distances and orientations were chosen to represent the surgical application (e.g. cholecystectomy). The experiments were conducted inside a surgical mannequin with no ambient light. Each imaging device used its own respective light source—external xenon fiber optic light source for the laparoscope and 2 ten candle white LEDs for the robotic camera. The video output from both systems is analog NTSC (National Television Systems Committee) composite. A Sensoray Model 2250 USB 2.0 frame grabber, connected to a laptop PC, was used to capture frames of video for later analysis.
MTF Testing
The modulation transfer function (MTF) is a widely used metric for performing quality evaluation of imaging systems. MTF is a measure of spatial resolution of an imaging system. MTF was used with the ISO 12233 Resolution chart to evaluate image quality. This chart was imaged with both the in vivo camera and laparoscope. The chart was parallel to the image sensor at a distance of 150 mm. Several still images were captured and analyzed. The Modulation Transfer Function is defined as:
where Mi and Mo are the modulation of the image and the modulation of the object, respectively. The modulation is defined as:
where Ymax is the maximum and Ymin is the minimum values of luminance. A plot of the MTF over all spatial frequencies defines the MTF of the system. MTF is calculated by computing the Fourier transform of the impulse response of the system. The impulse response is the response to a narrow line, which is the derivative of an edge response.
These MTF curves are plotted in
Color Accuracy
Color accuracy of the two systems was measured using a Macbeth ColorChecker™. The ColorChecker™ was placed in uniform illumination, and several still images were captured and the results were averaged over several still images. The test images were then converted to CIELAB color space by the Imatest™ application. The CIELAB space is based on human color perception. It is a three-dimensional space, where L* shows lightness, and (a*, b*) show color information. The CIELAB space was laid out to allow specification of color differences, in a linear manner. The Imatest program compares each test image color value to the known color value for each color patch in the target chart. The difference formula is given as:
ΔEab*=√{square root over ((ΔL*)2+(Δa*)2+(Δb*)2)}{square root over ((ΔL*)2+(Δa*)2+(Δb*)2)}{square root over ((ΔL*)2+(Δa*)2+(Δb*)2)} (3)
Plots of these color differences are shown in
Distortion
Distortion is an effect that causes straight lines to appear curved. Infinite series can be used to model lens distortion, which is a combination of radial and tangential components. However, usually only radial distortion needs to be considered, which can be modeled with one term. This can be modeled as:
ru=rd(1+κ1rd2) (4)
This equation relates the undistorted radius ru and the distorted radius rd. This one term model of distortion is referred to as barrel or pincushion distortion, depending on the sign of the parameter κ1. For these tests, the lower the value of κ1 the less distortion of the camera system.
An example of lens distortion for the laparoscope and in vivo camera is shown in
Discussion of Single Camera Comparison
In the MTF tests, the laparoscope had better results than the in vivo system. This is most likely caused by the limitation of lower quality optics in the in vivo system, since the MTF of the system is defined to be the product of the MTFs for each component of the system (lens, imager, etc). In the design of these devices, optics quality must be sacrificed for space, given the small physical size of the in vivo system. The laparoscope system is able to have higher quality optics, since the optics are not located in vivo and fiber optics instead lead from the laparoscope tip back to a high-precision optical instrument. This, however, does not mean that the laparoscope is superior to the in vivo robotic devices. The differences in spatial resolution may not be great enough to cause a subjective difference in the two systems. The in vivo robots described here significantly outperform conventional laparoscopes in distortion tests. The high amount of distortion in the laparoscope causes difficulty in quantitative area determinations during procedures. The in vivo robots do not suffer from these problems.
Ex Vivo Stereo Imaging Analysis
Stereoscopic display allows for the perception of depth and this can be extremely valuable in laparoscopic surgery. The robotic camera device shown in
The ex vivo stereo imaging experimental setup can be seen in
The geometry of the cameras is detailed in
The y coordinate of obj (and all points on the imaginary projection) given by yobj can be represented with the field of view angle θf, and the length of the line segment d.
Similarly, the value xmax is represented as
The x coordinate of the object is found using xmax and pixel n, the horizontal pixel position of obj.
The values of xobj and yobj can be used to find the object angle θobj. This substitution eliminates the unknown variable d.
Finally, the “slope” to the object, Sobj, is simply the arctangent of θobj.
Once the slope, Sobj, is found for the object in both of the stereoscopic cameras, the x and y position of the object can be determined.
The coordinate system for the object distance values, x and y, is centered at a point directly between the two cameras. This sets the x coordinate of the left and right cameras at −D/2 and D/2, respectfully. The line y=0 is the imaging plane of both cameras. Using the last equation, the “slope” to the object can be found for both the left and right cameras, SL and SR. IL and IR are the left and right y-intercepts where the camera “slopes” cross the system's y-axis.
y=SLx+IL (10)
y=SRx+IR (11)
Setting y=0 in each equation and using the known x coordinate (−D/2 and D/2) in each equation, IL and IR can be found:
The slope of each line is found from (9).
Setting x=xobj and y=yobj in (10) and (11) and solving for xobj leads to (15).
Similarly solving for yobj leads to (16).
yobj=SLxobj+IL=SRxobj+IR (16)
If the cameras are rotated, as they are in the in vivo imaging robot to provide a better view of the object, three new variables are introduced: θt, (the rotation angle of camera) and Δx and Δy (the shifts of the camera due to the rotation). Here, the rotation angle is assumed to be equal for both cameras. The new positions can be found using rotation matrices where
are vectors with the original slope.
The slopes in the rotated frame can then be determined from these rotated positions as shown in (19) and (20).
Using the shifts Δx and Δy, the new intercepts are found from (10) and (11):
Finally, the x and y coordinates are found by substituting the new slopes and intercepts into (15) and (16). To extend these results into three dimensions, the distance in the z direction is needed. The vertical slope can be determined using the following:
where θf is the vertical field of view, m is the vertical pixel position, and M is the total number of vertical pixels. The derivation of this is similar to the calculation of θobj in (5)-(9). The z component is found using the vertical slope Sv, and the distance to the object.
zreal=Sv·√{square root over (xobj2+yobj2)} (24)
The x coordinate remains the same (25).
xreal=xobj (25)
The y coordinate must be scaled by the cosine of the vertical angle (26).
yreal=yobj·cos(tan−1(Sv)) (26)
This mathematical analysis was implemented in the following section in an off-line Matlab program. Using recorded images, the object's positions were computed and plotted in space. Images are taken of objects of known dimensions to determine the accuracy of the stereo vision from the in vivo camera robot.
Testing of the Robotic Stereoscopic Camera Device
Using the experimental setup in
Pairs of corresponding points from the image pairs were analyzed and plotted. The shapes of the cylinders in the image can be reproduced in a depth map as shown in
Performing a Porcine Cholecystectomy with the Robotic Stereoscopic Camera Device
The in vivo camera robot was used to perform a porcine cholecystectomy (gall bladder removal). The surgeon used the video from the stereoscopic camera robot to perform the procedure. The three dimensional information was viewed by the surgeon using a stereoscopic display. Sample images are shown in
The surgeon performed the surgery in real time using the stereoscopic display. In addition, some captured images were post-processed to demonstrate the depth perception available to the surgeon. The resulting depth map for the images shown in
During the cholecystectomy, the animal was prepared as per normal procedure. Three small incisions were made in the pig's abdominal wall for the two tool ports and the laparoscope. The laparoscope was used to observe the procedure, but the surgeon used visual feed back from the in vivo stereoscopic camera. The in vivo stereoscopic robot was first inserted using a special trocar that allowed for the robot's electrical wire tethers. The remaining trocars were then placed and the abdomen was insufflated with carbon dioxide. Then the laparoscopic tools and laparoscope were inserted. A surgical assistant then lifted the in vivo robot into position on the abdominal wall using the magnetic holder and a laparoscopic tool as shown in
The operating surgeon then began the cholecystectomy, using the stereoscopic video feedback as with a standard laparoscopic surgical procedure. The cholecystectomy was performed using standard tools but with primary video feedback coming from the in vivo robot. After the cholecystectomy the in vivo robot was retracted by the tether.
Bench top tests were conducted to determine the torque that could be created with a robotic device similar to that device as depicted in
The second set of tests related to an examination of the kinematic configuration (i.e. joint motions) for the robot design, according to one embodiment. The robot is to manipulate tissue by applying forces with its end-effectors. This has to be done at a reasonable velocity. The endpoint forces and velocities that can be generated by a robot are highly dependent on the robot kinematics. Two possible, non-limiting configurations are shown in
One design, according to one embodiment, is shown schematically in
The two additional parameters required are the nominal speed that the robot can move its end-effectors, and the maximum endpoint force that can be applied by the end-effectors. In this example, the target endpoint force will be 3 N in all directions (x, y, and z) at every point in the workspace. The target endpoint velocity in this example will be 0.5 cm/second. Both of these parameters will vary throughout the robot's workspace. For example, the robot will be able to apply larger forces in the x direction when its “elbows” are straight. These parameters can be represented mathematically through the robot's Jacobian:
δx=Jδθ.
Here, the endpoint velocities, □x, are determined by the motors and actuators. They are the product of the joint velocities, □□ and the Jacobian matrix, J. The Jacobian contains the design parameters for joint lengths (ai) and joint configuration (□i).
For the proposed configuration, the Jacobian is given by:
where si=sin(□i) and ci=cos(□i). This equation will be used as part of the detailed design of each joint and link in the robot.
This application claims priority to U.S. Provisional Patent Application Ser. No. 60/815,741, filed Jun. 22, 2006, U.S. Provisional Patent Application Ser. No. 60/845,608, filed Sep. 19, 2006, U.S. Provisional Patent Application Ser. No. 60/868,030, filed Nov. 30, 2006, U.S. Provisional Patent Application Ser. No. 60/884,792, filed Jan. 12, 2007, and U.S. Provisional Patent Application Ser. No. 60/888,182, filed Feb. 5, 2007, all of which are hereby incorporated herein by reference in their entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 3870264 | Robinson | Mar 1975 | A |
| 3989952 | Timberlake et al. | Nov 1976 | A |
| 4246661 | Pinson | Jan 1981 | A |
| 4258716 | Sutherland | Mar 1981 | A |
| 4278077 | Mizumoto | Jul 1981 | A |
| 4538594 | Boebel et al. | Sep 1985 | A |
| 4568311 | Miyake | Feb 1986 | A |
| 4623183 | Aomori | Nov 1986 | A |
| 4736645 | Zimmer | Apr 1988 | A |
| 4771652 | Zimmer | Sep 1988 | A |
| 4852391 | Ruch et al. | Aug 1989 | A |
| 4896015 | Taboada et al. | Jan 1990 | A |
| 4897014 | Tietze | Jan 1990 | A |
| 4922755 | Oshiro et al. | May 1990 | A |
| 4990050 | Tsuge et al. | Feb 1991 | A |
| 5019968 | Wang et al. | May 1991 | A |
| 5108140 | Bartholet | Apr 1992 | A |
| 5172639 | Wiesman et al. | Dec 1992 | A |
| 5176649 | Wakabayashi | Jan 1993 | A |
| 5178032 | Zona et al. | Jan 1993 | A |
| 5187032 | Sasaki et al. | Feb 1993 | A |
| 5187796 | Wang et al. | Feb 1993 | A |
| 5195388 | Zona et al. | Mar 1993 | A |
| 5201325 | McEwen et al. | Apr 1993 | A |
| 5217003 | Wilk | Jun 1993 | A |
| 5263382 | Brooks et al. | Nov 1993 | A |
| 5271384 | McEwen et al. | Dec 1993 | A |
| 5284096 | Pelrine et al. | Feb 1994 | A |
| 5297443 | Wentz | Mar 1994 | A |
| 5297536 | Wilk | Mar 1994 | A |
| 5304899 | Sasaki et al. | Apr 1994 | A |
| 5307447 | Asano et al. | Apr 1994 | A |
| 5353807 | DeMarco | Oct 1994 | A |
| 5363935 | Schempf et al. | Nov 1994 | A |
| 5382885 | Salcudean et al. | Jan 1995 | A |
| 5388528 | Pelrine et al. | Feb 1995 | A |
| 5436542 | Petelin et al. | Jul 1995 | A |
| 5441494 | Oritz | Aug 1995 | A |
| 5458131 | Wilk | Oct 1995 | A |
| 5458583 | McNeely et al. | Oct 1995 | A |
| 5458598 | Feinberg et al. | Oct 1995 | A |
| 5471515 | Fossum et al. | Nov 1995 | A |
| 5515478 | Wang | May 1996 | A |
| 5524180 | Wang et al. | Jun 1996 | A |
| 5553198 | Wang et al. | Sep 1996 | A |
| 5562448 | Mushabac | Oct 1996 | A |
| 5588442 | Scovil et al. | Dec 1996 | A |
| 5620417 | Jang et al. | Apr 1997 | A |
| 5623582 | Rosenberg | Apr 1997 | A |
| 5624398 | Smith et al. | Apr 1997 | A |
| 5632761 | Smith et al. | May 1997 | A |
| 5645520 | Nakamura et al. | Jul 1997 | A |
| 5657429 | Wang et al. | Aug 1997 | A |
| 5657584 | Hamlin | Aug 1997 | A |
| 5674030 | Sigel | Oct 1997 | A |
| 5728599 | Rosteker et al. | Mar 1998 | A |
| 5736821 | Suyaman et al. | Apr 1998 | A |
| 5754741 | Wang et al. | May 1998 | A |
| 5762458 | Wang et al. | Jun 1998 | A |
| 5769640 | Jacobus et al. | Jun 1998 | A |
| 5791231 | Cohn et al. | Aug 1998 | A |
| 5792135 | Madhani et al. | Aug 1998 | A |
| 5797900 | Madhani et al. | Aug 1998 | A |
| 5807377 | Madhani et al. | Sep 1998 | A |
| 5815640 | Wang et al. | Sep 1998 | A |
| 5825982 | Wright et al. | Oct 1998 | A |
| 5841950 | Wang et al. | Nov 1998 | A |
| 5845646 | Lemelson | Dec 1998 | A |
| 5855583 | Wang et al. | Jan 1999 | A |
| 5876325 | Mizuno et al. | Mar 1999 | A |
| 5878193 | Wang et al. | Mar 1999 | A |
| 5878783 | Smart | Mar 1999 | A |
| 5895417 | Pomeranz et al. | Apr 1999 | A |
| 5906591 | Dario et al. | May 1999 | A |
| 5907664 | Wang et al. | May 1999 | A |
| 5911036 | Wright et al. | Jun 1999 | A |
| 5971976 | Wang et al. | Oct 1999 | A |
| 6001108 | Wang et al. | Dec 1999 | A |
| 6007550 | Wang et al. | Dec 1999 | A |
| 6030365 | Laufer | Feb 2000 | A |
| 6031371 | Smart | Feb 2000 | A |
| 6058323 | Lemelson | May 2000 | A |
| 6063095 | Wang et al. | May 2000 | A |
| 6066090 | Yoon | May 2000 | A |
| 6102850 | Wang et al. | Aug 2000 | A |
| 6107795 | Smart | Aug 2000 | A |
| 6132368 | Cooper | Oct 2000 | A |
| 6132441 | Grace | Oct 2000 | A |
| 6156006 | Brosens et al. | Dec 2000 | A |
| 6159146 | El Gazayerli | Dec 2000 | A |
| 6162171 | Ng et al. | Dec 2000 | A |
| D438617 | Cooper et al. | Mar 2001 | S |
| 6206903 | Ramans | Mar 2001 | B1 |
| D441076 | Cooper et al. | Apr 2001 | S |
| 6223100 | Green | Apr 2001 | B1 |
| D441862 | Cooper et al. | May 2001 | S |
| 6238415 | Sepetka et al. | May 2001 | B1 |
| 6240312 | Alfano et al. | May 2001 | B1 |
| 6241730 | Alby | Jun 2001 | B1 |
| 6244809 | Wang et al. | Jun 2001 | B1 |
| 6246200 | Blumenkranz et al. | Jun 2001 | B1 |
| D444555 | Cooper et al. | Jul 2001 | S |
| 6286514 | Lemelson | Sep 2001 | B1 |
| 6292678 | Hall et al. | Sep 2001 | B1 |
| 6293282 | Lemelson | Sep 2001 | B1 |
| 6296635 | Smith et al. | Oct 2001 | B1 |
| 6309397 | Julian et al. | Oct 2001 | B1 |
| 6309403 | Minoret et al. | Oct 2001 | B1 |
| 6312435 | Wallace et al. | Nov 2001 | B1 |
| 6321106 | Lemelson | Nov 2001 | B1 |
| 6327492 | Lemelson | Dec 2001 | B1 |
| 6331181 | Tierney et al. | Dec 2001 | B1 |
| 6346072 | Cooper | Feb 2002 | B1 |
| 6352503 | Matsui et al. | Mar 2002 | B1 |
| 6364888 | Niemeyer et al. | Apr 2002 | B1 |
| 6371952 | Madhani et al. | Apr 2002 | B1 |
| 6394998 | Wallace et al. | May 2002 | B1 |
| 6398726 | Ramans et al. | Jun 2002 | B1 |
| 6400980 | Lemelson | Jun 2002 | B1 |
| 6408224 | Okamoto et al. | Jun 2002 | B1 |
| 6424885 | Niemeyer et al. | Jul 2002 | B1 |
| 6432112 | Brock et al. | Aug 2002 | B2 |
| 6436107 | Wang et al. | Aug 2002 | B1 |
| 6441577 | Blumenkranz et al. | Aug 2002 | B2 |
| 6450104 | Grant et al. | Sep 2002 | B1 |
| 6451027 | Cooper et al. | Sep 2002 | B1 |
| 6454758 | Thompson et al. | Sep 2002 | B1 |
| 6459926 | Nowlin et al. | Oct 2002 | B1 |
| 6463361 | Wang et al. | Oct 2002 | B1 |
| 6468203 | Belson | Oct 2002 | B2 |
| 6468265 | Evans et al. | Oct 2002 | B1 |
| 6470236 | Ohtsuki | Oct 2002 | B2 |
| 6491691 | Morley et al. | Dec 2002 | B1 |
| 6491701 | Tierney et al. | Dec 2002 | B2 |
| 6493608 | Niemeyer et al. | Dec 2002 | B1 |
| 6496099 | Wang et al. | Dec 2002 | B2 |
| 6508413 | Bauer et al. | Jan 2003 | B2 |
| 6512345 | Borenstein | Jan 2003 | B2 |
| 6522906 | Salisbury, Jr. et al. | Feb 2003 | B1 |
| 6544276 | Azizi | Apr 2003 | B1 |
| 6548982 | Papanikolopoulos et al. | Apr 2003 | B1 |
| 6554790 | Moll | Apr 2003 | B1 |
| 6565554 | Niemeyer | May 2003 | B1 |
| 6574355 | Green | Jun 2003 | B2 |
| 6587750 | Gerbi et al. | Jul 2003 | B2 |
| 6591239 | McCall et al. | Jul 2003 | B1 |
| 6594552 | Nowlin et al. | Jul 2003 | B1 |
| 6610007 | Belson et al. | Aug 2003 | B2 |
| 6620173 | Gerbi et al. | Sep 2003 | B2 |
| 6642836 | Wang et al. | Nov 2003 | B1 |
| 6645196 | Nixon et al. | Nov 2003 | B1 |
| 6646541 | Wang et al. | Nov 2003 | B1 |
| 6648814 | Kim et al. | Nov 2003 | B2 |
| 6659939 | Moll et al. | Dec 2003 | B2 |
| 6661571 | Shioda et al. | Dec 2003 | B1 |
| 6671581 | Niemeyer et al. | Dec 2003 | B2 |
| 6676684 | Morley et al. | Jan 2004 | B1 |
| 6684129 | Salisbury, Jr. et al. | Jan 2004 | B2 |
| 6685648 | Flaherty et al. | Feb 2004 | B2 |
| 6685698 | Morley et al. | Feb 2004 | B2 |
| 6687571 | Byrne et al. | Feb 2004 | B1 |
| 6692485 | Brock et al. | Feb 2004 | B1 |
| 6699177 | Wang et al. | Mar 2004 | B1 |
| 6699235 | Wallace et al. | Mar 2004 | B2 |
| 6702734 | Kim et al. | Mar 2004 | B2 |
| 6702805 | Stuart | Mar 2004 | B1 |
| 6714839 | Salisbury, Jr. et al. | Mar 2004 | B2 |
| 6714841 | Wright et al. | Mar 2004 | B1 |
| 6719684 | Kim et al. | Apr 2004 | B2 |
| 6720988 | Gere et al. | Apr 2004 | B1 |
| 6726699 | Wright et al. | Apr 2004 | B1 |
| 6728599 | Wright et al. | Apr 2004 | B2 |
| 6730021 | Vassiliades, Jr. et al. | May 2004 | B2 |
| 6731988 | Green | May 2004 | B1 |
| 6746443 | Morley et al. | Jun 2004 | B1 |
| 6764441 | Chiel et al. | Jul 2004 | B2 |
| 6764445 | Ramans et al. | Jul 2004 | B2 |
| 6766204 | Niemeyer et al. | Jul 2004 | B2 |
| 6770081 | Cooper et al. | Aug 2004 | B1 |
| 6774597 | Borenstein | Aug 2004 | B1 |
| 6776165 | Jin | Aug 2004 | B2 |
| 6780184 | Tanrisever | Aug 2004 | B2 |
| 6783524 | Anderson et al. | Aug 2004 | B2 |
| 6785593 | Wang et al. | Aug 2004 | B2 |
| 6788018 | Blumenkranz | Sep 2004 | B1 |
| 6793653 | Sanchez et al. | Sep 2004 | B2 |
| 6799065 | Niemeyer | Sep 2004 | B1 |
| 6799088 | Wang et al. | Sep 2004 | B2 |
| 6801325 | Farr et al. | Oct 2004 | B2 |
| 6804581 | Wang et al. | Oct 2004 | B2 |
| 6810281 | Brock et al. | Oct 2004 | B2 |
| 6817972 | Snow | Nov 2004 | B2 |
| 6817974 | Cooper et al. | Nov 2004 | B2 |
| 6817975 | Farr et al. | Nov 2004 | B1 |
| 6820653 | Schempf et al. | Nov 2004 | B1 |
| 6824508 | Kim et al. | Nov 2004 | B2 |
| 6824510 | Kim et al. | Nov 2004 | B2 |
| 6832988 | Sprout | Dec 2004 | B2 |
| 6832996 | Woloszko et al. | Dec 2004 | B2 |
| 6836703 | Wang et al. | Dec 2004 | B2 |
| 6837846 | Jaffe et al. | Jan 2005 | B2 |
| 6837883 | Moll et al. | Jan 2005 | B2 |
| 6839612 | Sanchez et al. | Jan 2005 | B2 |
| 6840938 | Morley et al. | Jan 2005 | B1 |
| 6852107 | Wang et al. | Feb 2005 | B2 |
| 6858003 | Evans et al. | Feb 2005 | B2 |
| 6860346 | Burt et al. | Mar 2005 | B2 |
| 6860877 | Sanchez et al. | Mar 2005 | B1 |
| 6866671 | Tierney et al. | Mar 2005 | B2 |
| 6870343 | Borenstein et al. | Mar 2005 | B2 |
| 6871117 | Wang et al. | Mar 2005 | B2 |
| 6871563 | Choset et al. | Mar 2005 | B2 |
| 6879880 | Nowlin et al. | Apr 2005 | B2 |
| 6892112 | Wang et al. | May 2005 | B2 |
| 6899705 | Niemeyer | May 2005 | B2 |
| 6902560 | Morley et al. | Jun 2005 | B1 |
| 6905460 | Wang et al. | Jun 2005 | B2 |
| 6905491 | Wang et al. | Jun 2005 | B1 |
| 6911916 | Wang et al. | Jun 2005 | B1 |
| 6917176 | Schempf et al. | Jul 2005 | B2 |
| 6933695 | Blumenkranz | Aug 2005 | B2 |
| 6936001 | Snow | Aug 2005 | B1 |
| 6936003 | Iddan | Aug 2005 | B2 |
| 6936042 | Wallace et al. | Aug 2005 | B2 |
| 6943663 | Wang et al. | Sep 2005 | B2 |
| 6949096 | Davison et al. | Sep 2005 | B2 |
| 6951535 | Ghodoussi et al. | Oct 2005 | B2 |
| 6965812 | Wang et al. | Nov 2005 | B2 |
| 6974411 | Belson | Dec 2005 | B2 |
| 6974449 | Niemeyer | Dec 2005 | B2 |
| 6979423 | Moll | Dec 2005 | B2 |
| 6984203 | Tartaglia et al. | Jan 2006 | B2 |
| 6984205 | Gazdzinski | Jan 2006 | B2 |
| 6991627 | Madhani et al. | Jan 2006 | B2 |
| 6993413 | Sunaoshi | Jan 2006 | B2 |
| 6994703 | Wang et al. | Feb 2006 | B2 |
| 6994708 | Manzo | Feb 2006 | B2 |
| 6997908 | Carrillo, Jr. et al. | Feb 2006 | B2 |
| 7025064 | Wang et al. | Apr 2006 | B2 |
| 7027892 | Wang et al. | Apr 2006 | B2 |
| 7033344 | Imran | Apr 2006 | B2 |
| 7039453 | Mullick | May 2006 | B2 |
| 7042184 | Oleynikov et al. | May 2006 | B2 |
| 7048745 | Tierney et al. | May 2006 | B2 |
| 7053752 | Wang et al. | May 2006 | B2 |
| 7063682 | Whayne et al. | Jun 2006 | B1 |
| 7066879 | Fowler et al. | Jun 2006 | B2 |
| 7066926 | Wallace et al. | Jun 2006 | B2 |
| 7074179 | Wang et al. | Jul 2006 | B2 |
| 7077446 | Kameda et al. | Jul 2006 | B2 |
| 7083571 | Wang et al. | Aug 2006 | B2 |
| 7083615 | Peterson et al. | Aug 2006 | B2 |
| 7087049 | Nowlin et al. | Aug 2006 | B2 |
| 7090683 | Brock et al. | Aug 2006 | B2 |
| 7097640 | Wang et al. | Aug 2006 | B2 |
| 7105000 | McBrayer | Sep 2006 | B2 |
| 7107090 | Salisbury, Jr. et al. | Sep 2006 | B2 |
| 7109678 | Kraus et al. | Sep 2006 | B2 |
| 7118582 | Wang et al. | Oct 2006 | B1 |
| 7121781 | Sanchez et al. | Oct 2006 | B2 |
| 7125403 | Julian et al. | Oct 2006 | B2 |
| 7126303 | Farritor et al. | Oct 2006 | B2 |
| 7147650 | Lee | Dec 2006 | B2 |
| 7155315 | Niemeyer et al. | Dec 2006 | B2 |
| 7169141 | Brock et al. | Jan 2007 | B2 |
| 7182025 | Ghorbel et al. | Feb 2007 | B2 |
| 7182089 | Ries | Feb 2007 | B2 |
| 7199545 | Oleynikov et al. | Apr 2007 | B2 |
| 7206626 | Quaid, III | Apr 2007 | B2 |
| 7206627 | Abovitz et al. | Apr 2007 | B2 |
| 7210364 | Ghorbel et al. | May 2007 | B2 |
| 7214230 | Brock et al. | May 2007 | B2 |
| 7217240 | Snow | May 2007 | B2 |
| 7239940 | Wang et al. | Jul 2007 | B2 |
| 7250028 | Julian et al. | Jul 2007 | B2 |
| 7259652 | Wang et al. | Aug 2007 | B2 |
| 7273488 | Nakamura et al. | Sep 2007 | B2 |
| 7311107 | Harel et al. | Dec 2007 | B2 |
| 7339341 | Oleynikov et al. | Mar 2008 | B2 |
| 7372229 | Farritor et al. | May 2008 | B2 |
| 7447537 | Funda et al. | Nov 2008 | B1 |
| 7492116 | Oleynikov et al. | Feb 2009 | B2 |
| 7566300 | Devierre et al. | Jul 2009 | B2 |
| 7574250 | Niemeyer | Aug 2009 | B2 |
| 7637905 | Saadat et al. | Dec 2009 | B2 |
| 7645230 | Mikkaichi et al. | Jan 2010 | B2 |
| 7655004 | Long | Feb 2010 | B2 |
| 7670329 | Flaherty et al. | Mar 2010 | B2 |
| 7731727 | Sauer | Jun 2010 | B2 |
| 7762825 | Burbank et al. | Jul 2010 | B2 |
| 7772796 | Farritor et al. | Aug 2010 | B2 |
| 7785251 | Wilk | Aug 2010 | B2 |
| 7785333 | Miyamoto et al. | Aug 2010 | B2 |
| 7789825 | Nobis et al. | Sep 2010 | B2 |
| 7794494 | Sahatjian et al. | Sep 2010 | B2 |
| 7865266 | Moll et al. | Jan 2011 | B2 |
| 7960935 | Farritor et al. | Jun 2011 | B2 |
| 8179073 | Farritor et al. | May 2012 | B2 |
| 20010018591 | Brock et al. | Aug 2001 | A1 |
| 20010049497 | Kalloo et al. | Dec 2001 | A1 |
| 20020003173 | Bauer et al. | Jan 2002 | A1 |
| 20020026186 | Woloszka et al. | Feb 2002 | A1 |
| 20020038077 | de la Torre et al. | Mar 2002 | A1 |
| 20020065507 | Azizi | May 2002 | A1 |
| 20020091374 | Cooper | Jul 2002 | A1 |
| 20020103417 | Gazdzinski | Aug 2002 | A1 |
| 20020111535 | Kim et al. | Aug 2002 | A1 |
| 20020120254 | Julien et al. | Aug 2002 | A1 |
| 20020128552 | Nowlin et al. | Sep 2002 | A1 |
| 20020140392 | Borenstein et al. | Oct 2002 | A1 |
| 20020147487 | Sundquist et al. | Oct 2002 | A1 |
| 20020151906 | Demarais et al. | Oct 2002 | A1 |
| 20020156347 | Kim et al. | Oct 2002 | A1 |
| 20020171385 | Kim et al. | Nov 2002 | A1 |
| 20020173700 | Kim et al. | Nov 2002 | A1 |
| 20020190682 | Schempf et al. | Dec 2002 | A1 |
| 20030020810 | Takizawa et al. | Jan 2003 | A1 |
| 20030032855 | Shahinpoor | Feb 2003 | A1 |
| 20030045888 | Brock et al. | Mar 2003 | A1 |
| 20030065250 | Chiel et al. | Apr 2003 | A1 |
| 20030089267 | Ghorbel et al. | May 2003 | A1 |
| 20030092964 | Kim et al. | May 2003 | A1 |
| 20030097129 | Davison et al. | May 2003 | A1 |
| 20030100817 | Wang et al. | May 2003 | A1 |
| 20030114731 | Cadeddu et al. | Jun 2003 | A1 |
| 20030135203 | Wang et al. | Jul 2003 | A1 |
| 20030139742 | Wampler et al. | Jul 2003 | A1 |
| 20030144656 | Ocel et al. | Jul 2003 | A1 |
| 20030167000 | Mullick | Sep 2003 | A1 |
| 20030172871 | Scherer | Sep 2003 | A1 |
| 20030179308 | Zamorano et al. | Sep 2003 | A1 |
| 20030181788 | Yokoi et al. | Sep 2003 | A1 |
| 20030229268 | Uchiyama et al. | Dec 2003 | A1 |
| 20030230372 | Schmidt | Dec 2003 | A1 |
| 20040001750 | Kremerman | Jan 2004 | A1 |
| 20040024311 | Quaid | Feb 2004 | A1 |
| 20040034282 | Quaid | Feb 2004 | A1 |
| 20040034283 | Quaid | Feb 2004 | A1 |
| 20040034302 | Abovitz et al. | Feb 2004 | A1 |
| 20040050394 | Jin | Mar 2004 | A1 |
| 20040070822 | Shioda et al. | Apr 2004 | A1 |
| 20040099175 | Perrot et al. | May 2004 | A1 |
| 20040106916 | Quaid et al. | Jun 2004 | A1 |
| 20040111113 | Nakamura et al. | Jun 2004 | A1 |
| 20040138552 | Harel et al. | Jul 2004 | A1 |
| 20040140786 | Borenstein | Jul 2004 | A1 |
| 20040153057 | Davison | Aug 2004 | A1 |
| 20040173116 | Ghorbel et al. | Sep 2004 | A1 |
| 20040176664 | Iddan | Sep 2004 | A1 |
| 20040215331 | Chew et al. | Oct 2004 | A1 |
| 20040225229 | Viola | Nov 2004 | A1 |
| 20040254680 | Sunaoshi | Dec 2004 | A1 |
| 20040267326 | Ocel et al. | Dec 2004 | A1 |
| 20050014994 | Fowler et al. | Jan 2005 | A1 |
| 20050029978 | Oleynikov et al. | Feb 2005 | A1 |
| 20050043583 | Killmann et al. | Feb 2005 | A1 |
| 20050049462 | Kanazawa | Mar 2005 | A1 |
| 20050054901 | Yoshino | Mar 2005 | A1 |
| 20050054902 | Konno | Mar 2005 | A1 |
| 20050064378 | Toly | Mar 2005 | A1 |
| 20050065400 | Banik et al. | Mar 2005 | A1 |
| 20050083460 | Hattori et al. | Apr 2005 | A1 |
| 20050096502 | Khalili | May 2005 | A1 |
| 20050143644 | Gilad et al. | Jun 2005 | A1 |
| 20050154376 | Riviere et al. | Jul 2005 | A1 |
| 20050165449 | Cadeddu et al. | Jul 2005 | A1 |
| 20050283137 | Doyle et al. | Dec 2005 | A1 |
| 20050288555 | Binmoeller | Dec 2005 | A1 |
| 20050288665 | Woloszko | Dec 2005 | A1 |
| 20060020272 | Gildenberg | Jan 2006 | A1 |
| 20060046226 | Bergler et al. | Mar 2006 | A1 |
| 20060119304 | Farritor et al. | Jun 2006 | A1 |
| 20060149135 | Paz | Jul 2006 | A1 |
| 20060152591 | Lin | Jul 2006 | A1 |
| 20060155263 | Lipow | Jul 2006 | A1 |
| 20060195015 | Mullick et al. | Aug 2006 | A1 |
| 20060196301 | Oleynikov et al. | Sep 2006 | A1 |
| 20060198619 | Oleynikov et al. | Sep 2006 | A1 |
| 20060241570 | Wilk | Oct 2006 | A1 |
| 20060241732 | Denker et al. | Oct 2006 | A1 |
| 20060253109 | Chu | Nov 2006 | A1 |
| 20060258954 | Timberlake et al. | Nov 2006 | A1 |
| 20070032701 | Fowler et al. | Feb 2007 | A1 |
| 20070043397 | Ocel et al. | Feb 2007 | A1 |
| 20070055342 | Wu et al. | Mar 2007 | A1 |
| 20070080658 | Farritor et al. | Apr 2007 | A1 |
| 20070106113 | Ravo | May 2007 | A1 |
| 20070123748 | Meglan | May 2007 | A1 |
| 20070142725 | Hardin et al. | Jun 2007 | A1 |
| 20070156019 | Larkin et al. | Jul 2007 | A1 |
| 20070156211 | Ferren et al. | Jul 2007 | A1 |
| 20070167955 | De La Menardiere et al. | Jul 2007 | A1 |
| 20070225633 | Ferren et al. | Sep 2007 | A1 |
| 20070225634 | Ferren et al. | Sep 2007 | A1 |
| 20070241714 | Okeynikov et al. | Oct 2007 | A1 |
| 20070244520 | Ferren et al. | Oct 2007 | A1 |
| 20070250064 | Darois et al. | Oct 2007 | A1 |
| 20070255273 | Fernandez et al. | Nov 2007 | A1 |
| 20080004634 | Farritor et al. | Jan 2008 | A1 |
| 20080015565 | Davison | Jan 2008 | A1 |
| 20080015566 | Livneh | Jan 2008 | A1 |
| 20080033569 | Ferren et al. | Feb 2008 | A1 |
| 20080058835 | Farritor et al. | Mar 2008 | A1 |
| 20080058989 | Oleynikov et al. | Mar 2008 | A1 |
| 20080103440 | Ferren et al. | May 2008 | A1 |
| 20080109014 | de la Pena | May 2008 | A1 |
| 20080111513 | Farritor et al. | May 2008 | A1 |
| 20080119870 | Williams et al. | May 2008 | A1 |
| 20080132890 | Woloszko et al. | Jun 2008 | A1 |
| 20080164079 | Jacobsen | Jul 2008 | A1 |
| 20080183033 | Bern et al. | Jul 2008 | A1 |
| 20080221591 | Farritor et al. | Sep 2008 | A1 |
| 20080269557 | Marescaux et al. | Oct 2008 | A1 |
| 20090020724 | Paffrath | Jan 2009 | A1 |
| 20090024142 | Ruiz Morales | Jan 2009 | A1 |
| 20090048612 | Farritor et al. | Feb 2009 | A1 |
| 20090054909 | Farritor et al. | Feb 2009 | A1 |
| 20090069821 | Farritor et al. | Mar 2009 | A1 |
| 20090076536 | Rentschler et al. | Mar 2009 | A1 |
| 20090137952 | Ramamurthy et al. | May 2009 | A1 |
| 20090143787 | de la Pena | Jun 2009 | A9 |
| 20090163929 | Yeung et al. | Jun 2009 | A1 |
| 20090171373 | Farritor et al. | Jul 2009 | A1 |
| 20090234369 | Bax et al. | Sep 2009 | A1 |
| 20090236400 | Cole et al. | Sep 2009 | A1 |
| 20090240246 | Devill et al. | Sep 2009 | A1 |
| 20090247821 | Rogers | Oct 2009 | A1 |
| 20090248038 | Blumenkranz et al. | Oct 2009 | A1 |
| 20090281377 | Newell et al. | Nov 2009 | A1 |
| 20090305210 | Guru et al. | Dec 2009 | A1 |
| 20100010294 | Conlon et al. | Jan 2010 | A1 |
| 20100016659 | Weitzner et al. | Jan 2010 | A1 |
| 20100042097 | Newton et al. | Feb 2010 | A1 |
| 20100056863 | Dejima et al. | Mar 2010 | A1 |
| 20100069710 | Yamatani et al. | Mar 2010 | A1 |
| 20100069940 | Miller et al. | Mar 2010 | A1 |
| 20100081875 | Fowler et al. | Apr 2010 | A1 |
| 20100139436 | Kawashima et al. | Jun 2010 | A1 |
| 20100198231 | Scott | Aug 2010 | A1 |
| 20100204713 | Ruiz | Aug 2010 | A1 |
| 20100245549 | Allen et al. | Sep 2010 | A1 |
| 20100262162 | Omori | Oct 2010 | A1 |
| 20100318059 | Farritor et al. | Dec 2010 | A1 |
| 20110015569 | Kirschenman et al. | Jan 2011 | A1 |
| 20110020779 | Hannaford et al. | Jan 2011 | A1 |
| 20110071347 | Rogers et al. | Mar 2011 | A1 |
| 20110077478 | Freeman et al. | Mar 2011 | A1 |
| 20110152615 | Schostek et al. | Jun 2011 | A1 |
| 20110224605 | Farritor et al. | Sep 2011 | A1 |
| 20110230894 | Simaan et al. | Sep 2011 | A1 |
| 20110237890 | Farritor et al. | Sep 2011 | A1 |
| 20110238080 | Ranjit et al. | Sep 2011 | A1 |
| 20110270443 | Kamiya et al. | Nov 2011 | A1 |
| 20120035582 | Nelson et al. | Feb 2012 | A1 |
| 20120109150 | Quaid et al. | May 2012 | A1 |
| 20120253515 | Coste-Maniere et al. | Oct 2012 | A1 |
| 20130041360 | Farritor | Feb 2013 | A1 |
| 20130131695 | Scarfogliero et al. | May 2013 | A1 |
| Number | Date | Country |
|---|---|---|
| 2286756 | Feb 2011 | EP |
| 2004144533 | May 1990 | JP |
| 5115425 | May 1993 | JP |
| 200716235 | Jun 1993 | JP |
| 2006507809 | Sep 1994 | JP |
| 07 136173 | May 1995 | JP |
| 07306155 | Nov 1995 | JP |
| 7306155 | Nov 1995 | JP |
| 08-224248 | Sep 1996 | JP |
| 2003220065 | Aug 2003 | JP |
| 2004322310 | Jun 2004 | JP |
| 2004180781 | Jul 2004 | JP |
| 2004329292 | Nov 2004 | JP |
| 2006508049 | Mar 2006 | JP |
| WO 9221291 | May 1991 | WO |
| WO 02082979 | Oct 2002 | WO |
| 02100256 | Dec 2002 | WO |
| WO 02100256 | Dec 2002 | WO |
| WO 2005009211 | Feb 2005 | WO |
| WO 2006 005075 | Jan 2006 | WO |
| WO 2006079108 | Jan 2006 | WO |
| WO 2006052927 | May 2006 | WO |
| WO 2007111571 | Oct 2007 | WO |
| 2007149559 | Dec 2007 | WO |
| WO 2007149559 | Dec 2007 | WO |
| WO 2009023851 | Feb 2009 | WO |
| WO 2009144729 | Dec 2009 | WO |
| WO2010042611 | Apr 2010 | WO |
| WO2010046823 | Apr 2010 | WO |
| WO 2011118646 | Sep 2011 | WO |
| WO 2011135503 | Nov 2011 | WO |
| WO 2013009887 | Jan 2013 | WO |
| Entry |
|---|
| Abbou et al., “Laparoscopic Radical Prostatectomy with a Remote Controlled Robot.” The Journal of Urology. Jun. 2001, 165: 1964-1966. |
| Fraulob et al., “Miniature assistance module for robot-assisted heart surgely.” Biomed. Tech. 2002, 47 Suppl. 1, Pt. 1: 12-5. |
| Guber et al., “Miniaturized Instrument Systems for Minimally Invasive Diagnosis and Therapy.” Biomedizinische Technik. 2002, Band 47, Erganmngsband 1: 198-201. |
| Thomann et al., “The Design of a new type of Micro Robot for the Intestinal Inspection.” Proceedings of the 2002 IEEE/RSJIntl. Conference on Intelligent Robots and Systems EPFL. Oct. 2002: 1385-1390. |
| Guo et al., “Fish-like Underwater Microrobot with 3 DOF.” Proceedings of the 2002 IEEE International Conference on Robotics &Automation. May 2002: 738-743. |
| Fukuda et al., “Mechanism and Swimming Experiment of Micro Mobile Robot in Water.” Proceedings of the 1994 IEEE International Conference on Robotics and Automation. 1994: 814-. |
| Guo et al., “Micro Active Guide Wire Catheter System-Characteristic Evaluation, Electrical Model and Operability Evalu-ation of Micro Active Catheter,” Proceedings of the 1996 IEEE International Conference on Robotics and Automation. Apr. 1996: 2226-2231. |
| Yu et al., “Microrobotic Cell Injection.” Proceedings of the 2001 IEEE International Conference on Robotics & Automation. May 2001: 620-625. |
| Ruurda et al., “Robot-assisted surgical systems: a new era in laparoscopic surgely.”Ann. R. Coll. Surg. Engl. 2002, 84: 223-226. |
| Menciassi et al., “Robotic Solutions and Mechanisms for a Semi-Autonomous Endoscope.” Proceedings of the 2002 IEEEIRU Intl. Conference on Intelligent Robots and Systems EPFL. Oct. 2002: 1379-1384. |
| Ishiyama et al., “Spiral-type Micro-machine for Medical Applications.” 2000 International Symposium on Micromechatronic and Human Science. 2000: 65-69s. |
| Fearing et al., “Wing Transmission for a Micromechanical Flying Insect.” Proceedings of the 2000 IEEE International Conference on Robotics & Automation. Apr. 2000: 1509-1516. |
| Mei et al., “Wireless Drive and Control of a Swimming Microrobot.” Proceedings of the 2002 IEEE International Conference on Robotics & Automation. May 2002: 113 1-1136. |
| Yu, “M2A™ Capsule Endoscopy a Break-through Diagnostic Tool for Small Intestine Imaging,” vol. 25, No. 1, Gastroenterology Nursing, pp. 24-27 2001. |
| Strong et al., “Efficacy of Novel Robotic Camera vs a Standard Laparoscopic Camera,” Surgical Innovation, vol. 12, No. 4, Dec. 2005, Westminster Publications, Inc., pp. 315-318. |
| Worn et al., Esprit Project No. 33915: Miniaturized Robot for Micro Manipulation (Miniman) , Nov. 1998-Mar. 2002, http:||www,ipr,lra, uka.de/-microrobot/miniman/. |
| Meron, “The development of the swallowable video capsule (M2a),” Gastrointestinal Endoscopy 2000, 52, 6; 817-819. |
| Breda et al., “Future developments and perspectives in Lacroscopy”, Eur. Urology, 2001, 40, 1: 84-91. |
| Allendorf et al., “Postoperative Immune Function Varies Inversely with the Degree of Surgical Trauma in a Murine Model,” Surgical Endoscopy, 1997; 11:427-430. |
| Ang, “Active Tremor Compensation in Handheld Instrument for Microsurgery,” Doctoral dissertation, tech report CMU-RI-TR-04-28, Robotics Institute, Carnegie Mellon University, May 2004, 150 pp. |
| Atmel 80C5X2 Core, http://www.atmel.com, 2006. |
| Bailey et al., “Complications of Laparoscopic Surgery,” Quality Medical Publishers, Inc., 1995, 25 pp. |
| Ballantyne, “Robotic Surgery, Telerobotic Surgery, Telepresence, and Telementoring,” Surgical Endoscopy, 2002; 16:1389-1402. |
| Begos et al., “Laparoscopic Cholecystectomy: From Gimmick to Gold Standard,” J Clin Gastroenterol, 1994; 19(4):325-330. |
| Berg et al., “Surgery with Cooperative Robots,” Medicine Meets Virtual Reality, Feb. 2007. |
| Breedveld et al., “Locomotion through the Intestine by means of Rolling Stents,” Proceedings of the ASME Design Engineering Technical Conferences, 2004. |
| Calafiore et al., “Multiple Arterial Conduits Without Cardiopulmonary Bypass: Early Angiographic Results,” Ann Thorac Surg, 1999; 67:450-456. |
| Camarillo et al., “Robotic Technology in Surgery: Past, Present, and Future,” The American Journal of Surgery, 2004; 188: 2S-15S. |
| Cavuso{hacek over (g)}lu et al., “Robotics for Telesurgery: Second Generation Berkeley/UCSF Laparoscopic Telesurgical Workstation and Looking Towards the Future Applications,” Industrial Robot: An International Journal, 2003; 30(1):22-29. |
| Cavuso{hacek over (g)}lu et al., “Telesurgery and Surgical Simulation: Haptic Interfaces to Real and Virtual Surgical Environments.” In McLaughlin, M. L., Hespanha, J. P., and Sukhatme, G., editors. Touch in virtual environments. IMSC Series in Multimedia. 2001, 28 pp. |
| Choi et al., “Flexure-based Manipulator for Active Handheld Microsurgical Instrument,” Proceedings of the 27th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), Sep. 2005. |
| CrystalEyes, http://www.reald.com, 2007 (Stereo3D visualization for CAVEs, theaters and immersive environments), 1 pg. |
| Cuschieri, “Technology for Minimal Access Surgery,” BMJ, 1999, 319:1-6. |
| Stefanini et al., “Modeling and Experiments on a Legged Microrobot Locomoting in a Tubular, Compliant and Slippery Environment,” Int. Journal of Robotics Research, vol. 25, No. 5-6, pp. 551-560, May-Jun. 2006. |
| Dumpert et al., “Improving In Vivo Robot Vision Quality” in the Proceedings of Medicine Meets Virtual Reality, Long Beach, CA, Jan. 26-29, 2005. |
| Faraz et al., “Engineering Approaches to Mechanical and Robotic Design for Minimally Invasive Surgery (MIS),” Kluwer Academic Publishers (Boston), 2000, 13 pp. |
| Franklin et al., “Prospective Comparison of Open vs. Laparoscopic Colon Surgery for Carcinoma: Five-Year Results,” Dis Colon Rectum, 1996;39:S35-S46. |
| Fuller et al., “Laparoscopic Trocar Injuries: A Report from a U.S. Food and Drug Administration (FDA) Center for Devices and Radiological Health (CDRH) Systematic Technology Assessment of Medical Products (STAMP) Committee,” U.S. Food and Drug Administration, available at http://www.fda.gov, Finalized: Nov. 7, 2003; Updated: Jun. 24, 2005, 11 pp. |
| Glukhovsky et al., “The development and application of wireless capsule endoscopy,” Int. J.. Med. Robot. Comput. Assist. Surgery, 2004; 1 (1): 114-123. |
| Hanly et al., “Value of the Sages Learning Center in introducing new technology,” Surgical Endoscopy, 2004; 19(4): 477-483. |
| Hanly et al., “Robotic Abdominal Surgery,” The American Journal of Surgery, 2004; 188 (Suppl. to Oct. 1994): 19S-26S. |
| Heikkinen et al., “Comparison of laparoscopic and open Nissen fundoplication 2 Years after operation: A prospective randomized trial,” Surgical Endoscopy, 2000; 14:1019-1023. |
| Horgan et al., “Technical Report: Robots in Laparoscopic Surgery,” Journal of Laparoendoscopic & Advanced Surgical Techniques, 2001; 11(6):415-419. |
| Kim., “Early Experience with Telemanipulative Robot-Assisted Laparoscopic Cholecystectomy Using da Vinci,” Surgical Laparoscopy, Endoscopy & Percutaneous Techniques, 2002; 12(1):33-40. |
| Liem et al., “Comparison of Conventional Anterior Surgery and Laparoscopic Surgery for Inguinal-hernia Repair,” New England Journal of Medicine, 1997; 336 (22):1541-1547. |
| MacFarlane et al., “Force-Feedback Grasper Helps Restore the Sense of Touch in Minimally Invasive Surgery,” Journal of Gastrointestinal Surgery, 1999: 3: 278-285. |
| Mack, “Minimally Invasive and Robotic Surgery,” JAMA, Feb. 2001; 285(5):568-572. |
| Mack et al., “Present Role of Thoracoscopy in the Diagnosis and Treatment of Diseases of the Chest,” Ann Thorac Surgery, 1992; 54:403-409. |
| Menciassi et al., “Locomotion of a Legged Capsule in the Gastrointestinal Tract: Theoretical Study and Preliminary Technological Results,” IEEE Int. Conf. on Engineering in Medicine and Biology, San Francisco, CA, pp. 2767-2770, Sep. 2004. |
| Hissink, “Olympus Medical develops capsule camera technology,” Dec. 2004, accessed Aug. 29, 2007. http://www.letsgodigital.org , 3 pp. |
| Platt et al., “In Vivo Robotic Cameras can Enhance Imaging Capability During Laparoscopic Surgery,” in the Proceedings of the Society of American Gastrointestinal Endoscopic Surgeons (SAGES) Scientific Conference, Ft. Lauderdale, FL, Apr. 13-16, 2005, 1 pg. |
| Rentschler et al., “In vivo Robotics during the NEEMO 9 Mission,” Medicine Meets Virtual Reality, Feb. 2007, 1 pg. |
| Rentschler et al., “An In Vivo Mobile Robot for Surgical Vision and Task Assistance,” ASME Journal of Medical Devices, vol. 1, pp. 23-29, Mar. 2007. |
| Rentschler et al., “Mechanical Design of Robotic In Vivo Wheeled Mobility,” ASME Journal of Mechanical Design, 2006a, pp. 1-11, Accepted. |
| Rentschler et al., “Mobile In Vivo Camera Robots Provide Sole Visual Feedback for Abdominal Exploration and Cholecystectomy”, Journal of Surgical Endoscopy, 20-1: 135-138, 2006b. |
| Rentschler et al., “Modeling, Analysis, and Experimental Study of In Vivo Wheeled Robotic Mobility”, IEEE Transactions on Robotics, 22 (2): 308-321, 2006c. |
| Rentschler et al., “Natural Orifice Surgery with an Endoluminal Mobile Robot,” The Society of American Gastrointestinal Endoscopic Surgeons, Dallas, TX, Apr. 2006d. |
| Rentschler et al., “Mobile In Vivo Biopsy and Camera Robot”, Studies in Health and Informatics—Medicine Meets Virtual Reality, vol. 119., pp. 449-454, IOS Press, Long Beach, CA, 2006e. |
| Rentschler et al., “Toward In Vivo Wheeled Mobility,” Studies in Health Technology and Informatics—Medicine Meets Virtual Reality, ISO Press, Long Beach, CA, 2005a, 111:397-403. |
| Rentschler et al., “Mobile In Vivo Robots Can Assist in Abdominal Exploration” in the Proceedings of the Society of American Gastrointestinal Endoscopic Surgeons (SAGES) Scientific Conference, Ft. Lauderdale, FL, Apr. 13-16, 2005b. |
| Rentschler et al., “In Vivo Robots for Laparoscopic Surgery,” Studies in Health Technology and Informatics—Medicine Meets Virtual Reality, ISO Press, Newport Beach, CA, 2004a, 98:316-322. |
| Rosen et al., “The BlueDRAGON—A System of Measuring the Kinematics and the Dynamics of Minimally Invasive Surgical Tools In-Vivo,” Proc. of the 2002 IEEE International Conference on Robotics and Automation, Washington, DC, pp. 1876-1881, May 2002. |
| Rosen et al., “Objective Laparoscopic Skills Assessments of Surgical Residents Using Hidden Markov Models Based on Haptic Information and Tool/Tissue Interactions,” Studies in Health Technology and Informatics—Medicine Meets Virtual Reality, Jan. 2001. |
| Sackier et al., “Robotically assisted laparoscopic surgery,” Surgical Endoscopy, 1994; 8:63-6. |
| Satava, “Surgical Robotics: The Early Chronicles,” Surgical Laparoscopy, Endoscopy & Percutaneous Techniques, 2002; 12(1):6-16. |
| Schurr et al., “Robotics and Telemanipulation Technologies for Endoscopic Surgery,” Surgical Endoscopy, 2000; 14:375-381. |
| Stiff et al., “Long-term Pain: Less Common After Laparoscopic than Open Cholecystectomy,” British Journal of Surgery, 1994; 81:1368-1370. |
| Suzumori et al., “Development of Flexible Microactuator and its Applications to Robotics Mechanisms,” Proceedings of the IEEE International Conference on Robotics and Automation, 1991. |
| Taylor et al., “A Telerobotic Assistant for Laparoscopic Surgery,” IEEE Eng Med Biol, 1995; 279-87. |
| Tendick et al., “Applications of Micromechatronics in Minimally Invasive Surgery,” IEEE/ASME Transactions on Mechatronics, 1998; 3(1):34-42. |
| Way et al. , editors. “Fundamentals of Laparoscopic Surgery,” Churchill Livingstone Inc., 1995, 14 pp. |
| Oleynikov et al., In Vivo Robotic Laparoscopy, Surgical Innovation, Jun. 2005, 12(2): 177-181. |
| O'Neill, Surgeon takes new rout to gallbladder,: The Oregonian, Jun. 2007, 2 pp. |
| Rosen et al., “Force Controlled and Teleoperated Endoscopic, Grasper for Minimally Invasive Surgery—Experimental Performance Evaluation,”IEEE transactions of Biomedical Engineering, Oct. 1999; 46(10): pp. 1212-1221. |
| Salky, “What is the Penetration of Endoscopic Techniques into Surgical Practice?” Digestive Surgery 2000; 17:422-426. |
| Dakin et al., “Comparison of laparoscopic skills performance between standard instruments and two surfical robotic systems,” Surg Endosc., 2003 17:574-579. |
| Melvin et al., “Computer-Enhanced vs. Standard Laparoscopic Antireflux Surgery,” J Gastointest Surg 2002;6:11-16. |
| Park et al., “Trocar-less Instrumentation for Laparoscopy: Magnetic Positioning of Intra-abdominal Camera and Retractor,” Ann Surg, Mar. 2007;245(3):379-384. |
| Peters, “Mimimally Invasive Colectomy: Are the Potential Benefits Realized?” Dis Colon Rectom 1993; 36:751-756. |
| Gong et al., “Wireless endoscopy,” Gastrointestinal Endoscopy 2000; 51(6), pp. 725-729. |
| Bauer et al,. “Case Report: Remote Percutaneous Renal Access Using a New Automated Telesurgical Robotic System,” Telemedicine Journal and e-Health 2001; (4), pp. 341-347. |
| Kang et al., “Robotic Assistants Aid Surgeons During Minimally Invasive Procedures,” IEEE Engineering in Medicine and Biology, Jan./Feb. 2001, pp. 94-104. |
| Strong, “Efficacy of Novel Robotic Camera vs a Standard Laparoscopic Camera,” Surgical Innovation, Dec. 2005; 12(4), pp. 315-318. |
| Schwartz, “In the Lab: Robots that Slink and Squirm,” The New York Times, Mar. 27, 20074 pp. |
| Breedveld et al., “Design of Steerable Endoscopes to Improve the Visual Perception of Depth During Laparoscopic Surgery,” ASME , Jan. 2004, vol. 126, p. 1-5. |
| Fireman et al., “Diagnosing small bowel Crohn's disease with wireless capsule endoscopy,” Gut 2003; 52:390-392. |
| U.S. Appl. No. 12/192,663, filed Aug. 15, 2008. |
| U.S. Appl. No. 12/324,364, filed Nov. 26, 2008. |
| Applicant Response to Office Action dated Aug. 21, 2006, in related case U.S. Appl. No. 11/403,756, filed Nov. 21, 2006, 52 pp. |
| Applicant Response to Office Action dated Aug. 18, 2006, in related case U.S. Appl. No. 11/398,174, filed Nov. 7, 2006, 8 pp. |
| Examiner Interview Summary dated Nov. 30, 2006, in related case U.S. Appl. No. 11/398,174, 2 pp. |
| Applicant Response to Office Action dated Oct. 29, 2007, in related case U.S. Appl. No. 11/695,944, filed Jan. 22, 2008, 6 pp. |
| Applicant Amendment after Notice of Allowance under Rule 312, filed Aug. 25, 2008, in related case U.S. Appl. No. 11/695,944, 6 pp. |
| International Search Report and Written Opinion issued in PCT/US2008/069822, mailed Aug. 5, 2009, 10 pages. |
| Piers et al., “A miniature manipulator for integration in a self-propelling endoscope,” Sensors and Actuators 92 (1-3): 343-349, Aug. 2001. |
| International Search Report and Written Opinion of international application No. PCT/US2007/014567, mailed Apr. 28, 2008, 19 pp. |
| Rosen et al., Objective Evaluation of Laparoscopic Skills Based on Haptic Information and Tool/Tissue Interactions, Computer Aided Surgery, vol. 7, Issue 1, pp. 49-61 Jul. 2002. |
| Li et al. (2000), “Microvascular Anastomoses Performed in Rats Using a Microsurgical Telemanipulator,” Comp. Aid. Surg. 5:326-332. |
| Southern Surgeons Club (1991), “A prospective analysis of 1518 laparoscopic cholecystectomies,” N. Eng!. 1. Med. 324 (16): 1073-1078. |
| Wolfe et al. (1991), Endoscopic Cholecystectomy. An analysis of Complications. Arch. Surg. 126: I 192-1196. |
| Kazemier et al. (1998), “Vascular Injuries During Laparoscopy,” J. Am. Coli. Surg. 186(5):604-5. |
| Leggett et al. (2002), “Aortic injury during laparoscopic Fundoplication,” Surg. Endoscopy 16(2):362. |
| Munro, (2002), Laparoscopic access: complications, technologies, and techniques, Curro Opin. Obstet. Gynecol. 14 (4):365-74. |
| Tendick et al., (1993), “Sensing and Manipulation Problems in Endoscopic Surgery: Experiment, Analysis, and Observation,” Presence 2(1):66-81. |
| Chanthasopeephan et al., (2003), “Measuring Forces in Liver Cutting: New Equipment and Experimental Results,” Annals of Biomedical Engineering 31: 1372-1382. |
| Franzino, The Laprotek Surgical System and the Next Generation of Robotics Surg Clin North Am, 2003 83(6): 1317-1320. |
| Rentschler et al., “Miniature in vivo robotics for remote and harsh environments,” 2006, pp. 66-75. |
| Smart Pill “Fastastic Voyage: Smart Pill to Expand Testing,” http://www.smartpilldiagnostics.com, Apr. 13, 2005, 1 pg. |
| U.S. Appl. No. 11/947,097, filed Nov. 29, 2007, entitled “Robotic Devices With Agent Delivery Components and Related Methods,” 55 pp. |
| U.S. Appl. No. 11/966,741, filed Dec. 28, 2007, entitled “Methods Systems, and Devices for Surgical Visualization and Device Manipulation,” 52 pp. |
| Olympus, http://www.letsgodigital.org/en/news/articles/story—2150.html, 2004. |
| Patronik et al., “Development of a Tethered Epicardial Crawler for Minimally Invasive Cardiac Therapies,” IEEE, pp. 239-240. |
| Patronik et al., “Crawling on the Heart: A Mobile Robotic Device for Minimally Invasive Cardiac Interventions,” MICCAI, 2004, pp. 9-16. |
| Patronik et al., “Preliminary evaluation of a mobile robotic device for navigation and intervention on the beating heart,” Computer Aided Surgery, 10(4): 225-232, Jul. 2005. |
| Peirs et al., “A miniature manipulator for integration in a self-propelling endoscope,” Sensors and Actuators A, 2001, 92: 343-349. |
| Peters, “Minimally Invasive Colectomy: Are the Potential Benefits Realized?” Dis Colon Rectum 1993; 36: 751-756. |
| Phee et al., “Analysis and Development of Locomotion Devices for the Gastrointestinal Tract,” IEEE Transaction on Biomedical Engineering, vol. 49, No. 6, Jun. 2002, pp. 613-616. |
| Phee et al., “Development of Microrobotic Devices for Locomotion in the Human Gastrointestinal Tract,” International Conference on Computational Intelligence, Robotics and Autonomous Systems (CIRAS 2001), Nov. 28-30, 2001, Singapore. |
| Platt et al., “In Vivo Robotic Cameras can Enhance Imaging Capability During Laparoscopic Surgery,” in the Proceedings of the Society of American Gastrointestinal Endoscopic Surgeons (SAGES) Scientific Conference, Ft. Lauderdale, FL, Apr. 13-16, 2005, I pg. |
| Preliminary Amendment filed Apr. 11, 2007, in related case U.S. Appl. No. 11/403,756, 7 pp. |
| Preliminary Amendment filed Jul. 30, 2008, in related case U.S. Appl. No. 12/171,413, 4 pp. |
| RCE and Amendment filed Jun. 13, 2007, in related case U.S. Appl. No. 11/403,756, 8 pp. |
| Rentschler et al., “Mobile In Vivo Biopsy and Camera Robot,” Studies in Health and Infonnatics Medicine Meets Virtual Reality, vol. 119., pp. 449-454, IOS Press, Long Beach, CA, 2006e. |
| Rentschler et al., Mobile In Vivo Biopsy Robot, IEEE International Conference on Robotics and Automation, Orlando, Florida, May 2006, pp. 4155-4160. |
| Rentschler et al., “Miniature in vivo Robots for Remote and Harsh Environments,” IEEE Transactions on Information Technology in Biomedicine, Jan. 2006; 12(1): 66-75. |
| Rentschler et al., “An In Vivo Mobile Robot for Surgical Vision and Task Assistance,” Journal of Medical Devices, Mar. 2007, vol. 1: 23-29. |
| Rentschler et al., “In vivo Mobile Surgical Robotic Task Assistance,” 1 pg. |
| Rentschler et al., “In vivo Robotics during the NEEMO 9 Mission,” Medicine Meets Virtual Reality, Feb. 2007, I pg. |
| Rentschler et al., “In Vivo Robots for Laparoscopic Surgery,” Studies in Health Technology and Infonnatics—Medicine Meets Virtual Reality, ISO Press, Newport Beach, CA, 2004a, 98: 316-322. |
| Rentschler et al., “Mechanical Design of Robotic In Vivo Wheeled Mobility,” ASME Journal of Mechanical Design, 2006a, pp. 1-11. |
| Rentschler et al., “Mobile In Vivo Camera Robots Provide Sole Visual Feedback for Abdominal Exploration and Cholecystectomy,” Journal of Surgical Endoscopy, 20-I: 135-138, 2006b. |
| Rentschler et al., “Mobile In Vivo Robots Can Assist in Abdominal Exploration,” from the Proceedings of the Society of American Gastrointestinal Endoscopic Surgeons (SAGES) Scientific Conference, Ft. Lauderdale, FL, Apr. 13-16, 2005b. |
| Rentschler et al., “Modeling, Analysis, and Experimental Study of In Vivo Wheeled Robotic Mobility,” IEEE Transactions on Robotics, 22 (2): 308-321, 2005c. |
| Rentschler et al., “Natural Orifice Surgery with an Endoluminal Mobile Robot,” The Society of American Gastrointestinal Endoscopic Surgeons, Dallas, TX, Apr. 2006d, 14 pp. |
| Rentschler et al., “Theoretical and Experimental Analysis of In Vivo Wheeled Mobility,” ASME Design Engineering Technical Conferences: 28th Biennial Mechanisms and Robotics Conference, Salt Lake City, Utah, Sep. 28-Oct. 2, 2004, pp. 1-9. |
| Rentschler et al., “Toward In Vivo Mobility,” Studies in Health Technology and Infonnatics—Medicine Meets Virtual Reality, ISO Press, Long Beach, CA, 2005a, III: 397-403. |
| Response to Rule 312 Amendment in related case U.S. Appl. No. 11/695,944, dated Jan. 12, 2009, 2 pp. |
| Riviere et al., “Toward Active Tremor Canceling in Handheld Microsurgical Instruments,” IEEE Transactions on Robotics and Automation, Oct. 2003, 19(5): 793-800. |
| Rosen et al., “Force Controlled and Teleoperated Endoscopic, Grasper for Minimally Invasive Surgery—Experimental Performance Evaluation,” IEEE Transactions of Biomedical Engineering, Oct. 1999; 46(10): 1212-1221. |
| Rosen et al., “Objective Laparoscopic Skills Assessments of Surgical Residents Using Hidden Markov Models Based on Haptic Information and Tool/Tissue Interactions,” Studies in Health Technology and Informatics—Medicine Meets Virtual Reality, Jan. 2001, 7 pp. |
| Rosen et al., “Spherical Mechanism Analysis of a Surgical Robot for Minimally Invasive Surgery—Analytical and Experimental Approaches,” Studies in Health Technology and Informatics—Medicine Meets Virtual Reality, pp. 442-448, Jan. 2005. |
| Rosen et al., “Task Decomposition of Laparoscopic Surgery for Objective Evaluation of Surgical Residents' Learning Curve Using Hidden Markov Model,” Computer Aided Surgery, vol. 7, pp. 49-61, 2002. |
| Rosen et al., “The Blue DRAGON—A System of Measuring the Kinematics and the Dynamics of Minimally Invasive Surgical Tools In-Vivo,” Proc. of the 2002 IEEE International Conference on Robotics and Automation, Washington, DC, pp. 1876-1881, May 2002. |
| Ruurda et al., “Robot-Assisted surgical systems: a new era in laparoscopic surgery,” Ann R. Coll Surg Engl., 2002; 84: 223-226. |
| Ruurda et al., “Feasibility of Robot-Assisted Laparoscopic Surgery,” Surgical Laparoscopy, Endoscopy & Percutaneous Techniques, 2002; 12(1):41-45. |
| Sackier et al., “Robotically assisted laparoscopic surgery,” Surgical Endoscopy, 1994; 8: 63-66. |
| Salky, “What is the Penetration of Endoscopic Techniques into Surgical Practice?” Digestive Surgery, 2000; 17:422-426. |
| Satava, “Surgical Robotics: The Early Chronicles,” Surgical Laparoscopy, Endoscopy & Percutaneous Techniques, 2002; 12(1): 6-16. |
| Schippers et al., (1996) “Requirements and Possibilities of Computer-Assisted Endoscopic Surgery,” In: Computer Integrated Surgery: Technology and Clinical Applications, pp. 561-565. |
| Schurr et al., “Robotics and Telemanipulation Technologies for Endoscopic Surgery,” Surgical Endoscopy, 2000; 14: 375-381. |
| Schwartz, “In the Lab: Robots that Slink and Squirm,” The New York Times, Mar. 27, 2007, 4 pp. |
| Sharp LL-151-3D, http://www.sharp3d.com, 2006, 2 pp. |
| Slatkin et al., “The Development of a Robotic Endoscope,” Proceedings of the 1995 IEEE International Conference on Robotics and Automation, pp. 162-171, 1995. |
| Smart Pill “Fantastic Voyage: Smart Pill to Expand Testing,” http://www.smartpilldiagnostics.com, Apr. 13, 2005, 1 pg. |
| Southern Surgeons Club (1991), “A prospective analysis of 1518 laparoscopic cholecystectomies,” N. Eng. 1 Med. 324 (16): 1073-1078. |
| Stefanini et al., “Modeling and Experiments on a Legged Microrobot Locomoting in a Tubular Compliant and Slippery Environment,” Int. Journal of Robotics Research, vol. 25, No. 5-6, pp. 551-560, May-Jun. 2006. |
| Stiff et al.., “Long-term Pain: Less Common After Laparoscopic than Open Cholecystectomy,” British Journal of Surgery, 1994; 81: 1368-1370. |
| Strong, et al., “Efficacy of Novel Robotic Camera vs. a Standard Laproscopic Camera,” Surgical Innovation vol. 12, No. 4, Dec. 2005, Westminster Publications, Inc., pp. 315-318. |
| Suzumori et al., “Development of Flexible Microactuator and its Applications to Robotics Mechanisms,” Proceedings of the IEEE International Conference on Robotics and Automation, 1991: 1622-1627. |
| Taylor et al., “A Telerobotic Assistant for Laparoscopic Surgery,” IEEE Eng Med Biol, 1995; 279-287. |
| Tendick et al.. (1993), “Sensing and Manipulation Problems in Endoscopic Surgery: Experiment, Analysis, and Observation,” Presence 2( 1): 66-81. |
| International Preliminary Report on Patentability from related case PCT/US2007/014567, mailed Jan. 8, 2009, 11 pp. |
| International Search report and Written Opinion from international application No. PCT/US2012/41911, mailed Mar. 13, 2013. |
| International Search Report and Written Opinion from international application No. PCT/US12/46274, mailed Sep. 25, 2012. |
| International Search Report and Written Opinion from international application No. PCT/US2007/089191, mailed Nov. 10, 2008, 20 pp. |
| “International Search Report and Written Opinion from international application No. PCT/US07/14567, mailed Apr. 28, 2008, 19 pp.” |
| International Search Report and Written Opinion of international application No. PCT/US2008/069822, mailed Aug. 5, 2009, 12 pp. |
| International Search Report and Written Opinion of international application No. PCT/US2008/073334, mailed Jan. 12, 2009, 11 pp. |
| International Search Report and Written Opinion of international application No. PCT/US2008/073369, mailed Nov. 12, 2008, 12 pp. |
| International Search Report and Written Opinion issued in PCT/US11/46809, mailed Dec. 8, 2011. |
| Ishiyama et al., “Spiral-type Micro-machine for Medical Applications,” 2000 International Symposium on Micromechatronics and Human Science, 2000: 65-69. |
| Jagannath et al., “Peroral transgastric endoscopic ligation of fallopian tubes with long-term survival in a porcine model,” Gastrointestinal Endoscopy, 2005; 61(3): 449-453. |
| Kalloo et al., “Flexible transgastric peritoneoscopy: a novel approach to diagnostic and therapeutic interventions in the peritoneal cavity,” Gastrointestinal Endoscopy, 2004; 60(1): 114-117. |
| Kang et al., “Robotic Assistants Aid Surgeons During Minimally Invasive Procedures,” IEEE Engineering in Medicine and Biology, Jan.-Feb. 2001; pp. 94-104. |
| Kantsevoy et al., “Endoscopic gastrojejunostomy with survival in a porcine model,” Gastrointestinal Endoscopy, 2005; 62(2): 287-292. |
| Kantsevoy et al., “Transgastric endoscopic splenectomy,” Surgical Endoscopy, 2006; 20: 522-525. |
| Kazemier et al. (1998), “Vascular Injuries During Laparoscopy,” J. Am. Coli. Surg. 186(5): 604-5. |
| Kim, “Early Experience with Telemanipulative Robot-Assisted Laparoscopic Cholecystectomy Using da Vinci,” Surgical Laparoscopy, Endoscopy & Percutaneous Techniques, 2002; 12(1):33-40. |
| Ko et al., “Per-Oral transgastric abdominal surgery,” Chinese Journal of Digestive Diseases, 2006; 7: 67-70. |
| Lafullarde et al., “Laparoscopic Nissen Fundoplication: Five-year Results and Beyond,” Arch/Surg, Feb. 2001; 136:180-184. |
| Leggett et al. (2002), “Aortic injury during laparoscopic fundoplication,” Surg. Endoscopy 16(2): 362. |
| Li et al. (2000), “Microvascular Anastomoses Performed in Rats Using a Microsurgical Telemanipulator,” Comp. Aid. Surg. 5: 326-332. |
| Liem et al., “Comparison of Conventional Anterior Surgery and Laparoscopic Surgery for Inguinal-hernia Repair,” New England Journal of Medicine, 1997; 336 (22): 1541-1547. |
| Macfarlane et al., “Force-Feedback Grasper Helps Restore the Sense of Touch in Minimally Invasive Surgery,” Journal of Gastrointestinal Surgery, 1999; 3: 278-285. |
| Mack et al., “Present Role of Thoracoscopy in the Diagnosis and Treatment of Diseases of the Chest,” Ann Thorac Surgery, 1992; 54: 403-409. |
| Mack, “Minimally Invasive and Robotic Surgery,” JAMA, Feb. 2001; 285(5): 568-572. |
| Mei et al., “Wireless Drive and Control of a Swimming Microrobot,” Proceedings of the 2002 IEEE International Conference on Robotics & Automation, May 2002: 1131-1136. |
| Melvin et al., “Computer-Enhanced vs. Standard Laparoscopic Antireflux Surgery,” J Gastrointest Surg 2002; 6: 11-16. |
| Menciassi et al., “Locomotion of a Leffed Capsule in the Gastrointestinal Tract: Theoretical Study and Preliminary Technological Results,” IEEE Int. Conf. on Engineering in Medicine and Biology, San Francisco, CA, pp. 2767-2770, Sep. 2004. |
| Menciassi et al., “Robotic Solutions and Mechanisms for a Semi-Autonomous Endoscope,” Proceedings of the 2002 IEEE/RSJ Intl. Conference on Intelligent Robots and Systems, Oct. 2002; 1379-1384. |
| Menciassi et al., “Shape memory alloy clamping devices of a capsule for monitoring tasks in the gastrointestinal tract,” J. Micromech. Microeng, 2005, 15: 2045-2055. |
| Meron, “The development of the swallowable video capsule (M2A),” Gastrointestinal Endoscopy 2000; 52 6: 817-819. |
| Micron, http://www.micron.com, 2006, ¼-inch VGA NTSC/PAL CMOS Digital Image Sensor, 98 pp. |
| Midday Jeff et al., “Material Handling System for Robotic natural Orifice Surgery”, Proceedings of the 2011 Design of medical Devices Conference, Apr. 12-14, 2011, Minneapolis, MN, 4 pages. |
| Miller, Ph.D., et al., “In-Vivo Stereoscopic Imaging System with 5 Degrees-of-Freedom for Minimal Access Surgery,” Dept. of Computer Science and Dept. of Surgery, Columbia University, New York, NY, 7 pp. |
| Munro (2002), “Laparoscopic access: complications, technologies, and techniques,” Curro Opin. Obstet. Gynecol., 14(4): 365-74. |
| Nio et al., “Efficiency of manual vs robotical (Zeus) assisted laparoscopic surgery in the performance of standardized tasks,” Surg Endosc, 2002; 16: 412-415. |
| Office Action dated Apr. 17, 2007, received in related case U.S. Appl. No. 11/552,379, 5 pp. |
| Office Action dated Apr. 3, 2009, received in related case U.S. Appl. No. 11/932,516, 43 pp. |
| Office Action dated Aug. 18, 2006, received in related case U.S. Appl. No. 11/398,174, 6 pp. |
| Office Action dated Aug. 21, 2006, received in related case U.S. Appl. No. 11/403,756, 6 pp. |
| Office Action dated Oct. 29, 2007, received in related case U.S. Appl. No. 11/695,944, 6 pp. |
| Office Action dated Oct. 9, 2008, received in related case U.S. Appl. No. 11/932,441, 4 pp. |
| Oleynikov et al., “In Vivo Camera Robots Provide Improved Vision for Laparoscopic Surgery,” Computer Assisted Radiology and Surgery (CARS), Chicago, IL, Jun. 23-26, 2004b. |
| Oleynikov et al., “In Vivo Robotic Laparoscopy,” Surgical Innovation, Jun. 2005, 12(2): 177-181. |
| Oleynikov et al., “Miniature Robots Can Assist in Laparoscopic Cholecystectomy,” Journal of Surgical Endoscopy, 19-4: 473-476, 2005. |
| O'Neill, “Surgeon takes new route to gallbladder,” The Oregonian, Jun. 2007, 2 pp. |
| Orlando et al., (2003), “Needle and Trocar Injuries in Diagnostic Laparoscopy under Local Anesthesia: What Is the True Incidence of These Complications?” Journal of Laparoendoscopic & Advanced Surgical Techniques 13(3): 181-184. |
| Park et al., “Trocar-less Instrumentation for Laparoscopy: Magnetic Positioning of Intra-abdominal Camera and Retractor,” Ann Surg, Mar. 2007; 245(3): 379-384. |
| Park et al., “Experimental studies of transgastric gallbladder surgery: cholecystectomy and cholecystogastric anastomosis (videos),” Gastrointestinal Endoscopy, 2005; 61(4): 601-606. |
| Tendick et al., “Applications of Micromechatronics in Minimally Invasive Surgery,” IEEE/ASME Transactions on Mechatronics, 1998; 3(1): 34-42. |
| Thomann et al., “The Design of a new type of Micro Robot for the Intestinal Inspection,” Proceedings of the 2002 IEEE Intl. Conference on Intelligent Robots and Systems, Oct. 2002: 1385-1390. |
| U.S. Appl. No. 60/180,960, filed Feb. 2000. |
| U.S. Appl. No. 60/956,032, filed Aug. 15, 2007. |
| U.S. Appl. No. 60/983,445, filed Oct. 29, 2007. |
| U.S. Appl. No. 60/990,062, filed Nov. 26, 2007. |
| U.S. Appl. No. 60/990,076, filed Nov. 26, 2007. |
| U.S. Appl. No. 60/990,086, filed Nov. 26, 2007. |
| U.S. Appl. No. 60/990,106, filed Nov. 26, 2007. |
| U.S. Appl. No. 60/990,470, filed Nov. 27, 2007. |
| U.S. Appl. No. 61/025,346, filed Feb. 1, 2008. |
| U.S. Appl. No. 61/030,588, filed Feb. 22, 2008. |
| U.S. Appl. No. 61/030,617, filed Feb. 22, 2008. |
| Way et al., (editors), “Fundamentals of Laparoscopic Surgery,” Churchill Livingstone Inc., 1995, 14 pp. |
| Wolfe et al., “Endoscopic Cholecystectomy: An analysis of Complications,” Arch. Surg. Oct. 1991; 126: 1192-1196. |
| Worn et al., “Espirit Project No. 33915: Miniaturised Robot for Micro Manipulation (MINIMAN)”, Nov. 1998; http://www.ipr.ira.ujka.de/-microbot/miniman. |
| Yu et al., “Microrobotic Cell Injection,” Proceedings of the 2001 IEEE International Conference on Robotics and Automation, May 2001; 620-625. |
| Yu, BSN, RN, “M2ATM Capsule Endoscopy A Breakthrough Diagnostic Tool for Small Intestine Imagining,” vol. 25, No. 1, Gastroenterology Nursing, pp. 24-27. |
| International Search Report and Written Opinion of international application No. PCT/US2010/061137, mailed Feb. 11, 2011, 10 pp. |
| Abbou et al., “Laparoscopic Radical Prostatectomy with a Remote Controlled Robot,” The Journal of Urology, Jun. 2001, 165: 1964-1966. |
| Glukhovsky et al.., “The development and application of wireless capsule endoscopy,” Int. J. Med. Robot. Comput. Assist. Surgery, 2004; I (1): 114-123. |
| Gong et al., Wireless endoscopy, Gastrointestinal Endoscopy 2000; 51(6): 725-729. |
| Hanly et al., “Value of the SAGES Learning Center in introducing new technology,” Surgical Endoscopy, 2004; 19 (4): 477-483. |
| Abbott et al., “Design of an Endoluminal Notes Robotic System,” from the Proceedings of the 2007 IEEE/RSJ Int'l Conf. on Intelligent Robot Systems, San Diego, CA, Oct. 29-Nov. 2, 2007, pp. 410-416. |
| Allendorf et al., “Postoperative Immune Function Varies Inversely with the Degree of Surgical Trauma in a Murine Model,” Surgical Endoscopy 1997; 11:427-430. |
| Ang, “Active Tremor Compensation in Handheld Instrument for Microsurgery,” Doctoral Dissertation, tech report CMU-RI-TR-04-28, Robotics Institute, Carnegie Mellon Unviersity, May 2004, 167pp. |
| Applicant Amendment after Notice of Allowance under Rule 312, filed Aug. 25, 2008, in related case U.S. Appl. No. 11/695,944, 6pp. |
| Applicant Response to Office Action dated Apr. 17, 2007, in related case U.S. Appl. No. 11/552,379, filed Aug. 8, 2007, 7 pp. |
| Applicant Response to Office Action dated Aug. 18, 2006, in related case U.S. Appl. No. 11/398,174, filed Nov. 7, 2006, 8pp. |
| Applicant Response to Office Action dated Aug. 21, 2006, in related case U.S. Appl. No. 11/403,756, filed Nov. 21, 2006, 52pp. |
| Applicant Response to Office Action dated Oct. 29, 2007, in related case U.S. Appl. No. 11/695,944, filed Jan. 22, 2008, 6pp. |
| Atmel 80C5X2 Core, http://www.atmel.com, 2006, 186pp. |
| Bailey et al., “Complications of Laparoscopic Surgery,” Quality Medical Publishers, Inc., 1995, 25pp. |
| Ballantyne, “Robotic Surgery, Telerobotic Surgery, Telepresence, and Telementoring,” Surgical Endoscopy, 2002; 16: 1389-1402. |
| Bauer et al., “Case Report: Remote Percutaneous Renal Percutaneous Renal Access Using a New Automated Telesurgical Robotic System,” Telemedicine Journal and e-Health 2001; (4): 341-347. |
| Begos et al., “Laparoscopic Cholecystectomy: From Gimmick to Gold Standard,” J Clin Gastroenterol, 1994; 19(4): 325-330. |
| Berg et al., “Surgery with Cooperative Robots,” Medicine Meets Virtual Reality, Feb. 2007, 1 pg. |
| Breda et al., “Future developments and perspectives in laparoscopy,” Eur. Urology 2001; 40(1): 84-91. |
| Breedveld et al., “Design of Steerable Endoscopes to Improve the Visual Perception of Depth During Laparoscopic Surgery,” ASME, Jan. 2004; vol. 126, pp. 1-5. |
| Breedveld et al., “Locomotion through the Intestine by means of Rolling Stents,” Proceedings of the ASME Design Engineering Technical Conferences, 2004, pp. 1-7. |
| Calafiore et al., Multiple Arterial Conduits Without Cardiopulmonary Bypass: Early Angiographic Results,: Ann Thorac Surg, 1999; 67: 450-456. |
| Camarillo et al., “Robotic Technology in Surgery: Past, Present and Future,” The American Journal of Surgery, 2004; 188: 2S-15. |
| Cavusoglu et al., “Telesurgery and Surgical Simulation: Haptic Interfaces to Real and Virtual Surgical Environments,” In McLaughliin, M.L., Hespanha, J.P., and Sukhatme, G., editors. Touch in virtual environments, IMSC Series in Multimedia 2001, 28pp. |
| Cavusoglu et al., “Robotics for Telesurgery: Second Generation Berkeley/UCSF Laparoscopic Telesurgical Workstation and Looking Towards the Future Applications,” Industrial Robot: An International Journal, 2003; 30(1): 22-29. |
| Chanthasopeephan et al., (2003), “Measuring Forces in Liver Cutting: New Equipment and Experimenal Results,” Annals of Biomedical Engineering 31: 1372-1382. |
| Choi et al., “Flexure-based Manipulator for Active Handheld Microsurgical Instrument,” Proceedings of the 27th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS), Sep. 2005, 4pp. |
| Cuschieri, “Technology for Minimal Access Surgery,” BMJ, 1999, 319: 1-6. |
| Dakin et al., “Comparison of laparoscopic skills performance between standard instruments and two surgical robotic systems,” Surg Endosc., 2003; 17: 574-579. |
| Dumpert et al., “Improving in Vivo Robot Visioin Quality,” from the Proceedings of Medicine Meets Virtual Realtiy, Long Beach, CA, Jan. 26-29, 2005. 1 pg. |
| Dumpert et al., “Stereoscopic In Vivo Surgical Robots,” IEEE Sensors Special Issue on In Vivo Sensors for Medicine, Jan. 2007, 10 pp. |
| Examiner Interview Summary dated Aug. 6 and Aug. 12, 2008, in related case U.S. Appl. No. 11/695,944, 1 pg. |
| Examiner Interview Summary dated May 9, 2008, in related case U.S. Appl. No. 11/695,944, 1 pg. |
| Examiner Interview Summary dated Nov. 30, 2006, in related case U.S. Appl. No. 11/398,174, 2pp. |
| Falcone et al., “Robotic Surgery,” Clin. Obstet. Gynecol. 2003, 46(1): 37-43. |
| Faraz et al., “Engineering Approaches to Mechanical and Robotic Design for Minimaly Invasive Surgery (MIS),” Kluwer Academic Publishers (Boston), 2000, 13pp. |
| Fearing et al., “Wing Transmission for a Micromechanical Flying Insect,” Proceedings of the 2000 IEEE International Conference to Robotics & Automation, Apr. 2000; 1509-1516. |
| Fireman et al., “Diagnosing small bowel Crohn's desease with wireless capsule endoscopy,” Gut 2003; 52: 390-392. |
| Flynn et al., “Tomorrow's Surgery: micromotors and microbots for minimally invasive procedures,” Minimally Invasive Surgery & Allied Technologies. |
| Franklin et al., “Prospective Comparison of Open vs. Laparoscopic Colon Surgery for Carcinoma: Five-Year Results,” Dis Colon Rectum, 1996; 39: S35-S46. |
| Franzino, “The Laprotek Surgical System and the Next Generation of Robotics,” Surg Clin North Am, 2003 83(6). |
| Fraulob et al., “Miniature assistance module for robot-assisted heart surgery,” Biomed. Tech. 2002, 47 Suppl. 1, Pt. 1:12-15. |
| Fukuda et al., “Mechanism and Swimming Experiment of Micro Mobile Robot in Water,” Proceedings of the 1994 IEEE International Conference on Robotics and Automation, 1994: 814-819. |
| Fukuda et al., “Micro Active Catheter System with Multi Degrees of Freedom,” Proceedings of the IEEE International Conference on Robotics and Automation, May 1994, pp. 2290-2295. |
| Fuller et al., “Laparoscopic Trocar Injuries: A Report from a U.S. Food and Drug Administration (FDA) Center for Devices and Radiological Health (CDRH) Systematic Technology Assessment of Medical Products (STAMP) Committe,” U.S. Food and Drug Adminstration, available at http://www.fdaJ:?;ov, Finalized: Nov. 7, 2003; Updated: Jun. 24, 2005, 11 pp. |
| Grady, “Doctors Try New Surgery for Gallbladder Removal,” The New York Times, Apr. 20, 2007, 3 pp. |
| Guber et al., “Miniaturized Instrumetn Systems for Minimally Invasive Diagnosis and Therapy,” Biomedizinishe Technic. 2002, Band 47, Erganmngsband 1. |
| Palm, William, “Rapid Prototyping Primer” May 1998 (revised Jul. 30, 2002) (http://www.me.psu.edu/lamancusa/rapidpro/primer/chapter2.htm). |
| Number | Date | Country | |
|---|---|---|---|
| 20080004634 A1 | Jan 2008 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60815741 | Jun 2006 | US | |
| 60845608 | Sep 2006 | US | |
| 60868030 | Nov 2006 | US | |
| 60884792 | Jan 2007 | US | |
| 60888182 | Feb 2007 | US |
