Patent Application
20240350122
The present disclosure relates to a diagnostic imaging system and a method of visualising diagnostic images by facilitating control of the diagnostic system from a remote location.
Any references to methods, apparatus or documents of the prior art are not to be taken as constituting any evidence or admission that they formed, or form part of the common general knowledge.
Robotics started to solve medical challenges in the 1940s, but their utility was only realised in the early 1980s from industrial systems. Over the past few decades, robots have grown in precision and complexity for various medical applications. Robots have significant advantages in delivering more precise manoeuvres in the examination room, reducing unintended damage and shortened examination times for patients. The primary role of robots is to improve the safety, success, and consistency of medical procedures. Modern medical robots present in many forms, with applications ranging from complex joint replacements to rehabilitation. Most robotic systems have multiple integrated sensors to measure, track, align, and analyse the patient and environment. Some sensors are invasive and installed on or in patients, while others (e.g., electro-optical cameras) are non-invasive.
Medical support through robotics is developing rapidly due to the increasing demand for remote capabilities and alleviating the stresses on the operator. Progress has been made in autonomous surgical manoeuvres, optical coherence, tomography guidance, worldwide high-speed data connectivity and motion compensation, revolutionising remote surgical control applications. However, research has overlooked these technological advantages in the context of autonomous remote cardiac ultrasound imaging applications. To the best of our knowledge, there is currently no commercially implemented ultrasound imaging system which allows robots to autonomously perform the ultrasound imaging, using the same movements as sonographers on a patient, and interpret the scanned image to deliver a cardiac diagnosis.
Ultrasound scanning is a low-risk, low-impact medical procedure that, with full automation, could significantly reduce the cost to ultrasound scanning organisations and lower the impact on sonographers. Additionally, the synergy between medical and robotic technologies could improve access to cardiac ultrasound imaging to remote communities, which previously did not have such facilities-ultimately saving lives. It is desirable to provide a robotic system in combination with ultrasound diagnostic equipment to reduce the impact on sonographers locally, and support remote cardiac imaging operated by a remotely located sonographer that is comparable to an in-room scan.
In an aspect, the invention provides a diagnostic imaging system comprising:
In an embodiment, selection of any one of the said plurality of modes positions the elements of the manipulator arm into a predetermined starting position corresponding to the selected mode before allowing movement of the one or more of the joints of the manipulator arm in selected mode, by using the controller, within the pre-defined range for the selected mode to limit contact of the probe module within the working region on the patient's body that corresponds to the selected operating mode.
In an embodiment, switching between a first and second operable mode, based on a selection on the control panel, results in the positioning of the elements of the manipulator arm into the predetermined starting position for the first operable mode before effecting further movement and positioning of the elements of the manipulator to the predetermined starting position for the second operable mode thereby preventing collision between the subject's anatomy and the diagnostic probe when switching between the first and second operable modes.
In an embodiment, selection of each operable mode corresponds to a respective set of spatial limits on movement of the joints of the manipulator during operation in the selected mode.
In an embodiment, the diagnostic imaging system further comprises a user interface being provided on said control panel, control panel being in communication with the controller, the user interface comprising a display device to display visual representation of the plurality of operable modes, the input interface being configured to receive input from the user to effect selection of one of the operable modes.
In an embodiment, the display device is configured to present a visual or audio representation of a force being applied by the ultrasound probe on the subject's anatomy during use in each operable mode.
In an embodiment, the system further comprises a sensor positioned relative to the one or more elements of the manipulator arm for sensing force applied by manipulator arm on the subject, based on controlling input received by control from a user during operation in a selected operable mode wherein the sensor is coupled with a force feedback module that communicates with the controller and the coupling arrangement to limit or effect movement of the elements of the manipulator and at least partially override the controlling input provided by the user when the force sensed by the sensor exceeds a preset threshold value.
In an embodiment, the controller further comprises a feedback system to allow a user to operate the manipulator arm using haptic, audio or visual feedback.
Preferably, for each operable mode, the feedback controller is operated with a corresponding set of operational parameters to apply a specific scale factor and direction control to the feedback controller for each operable mode.
In an embodiment, during operation in any of the operating modes, the spatial position of the ultrasound probe can apply a constant force on the subject's anatomy during use at a specific position of interest, or during probe motion where it is necessary to follow the body contours of the patient.
In another aspect, the diagnostic imaging system comprises an additional body manipulator arm, the body manipulator arm comprising a base that is movably attached to a bed frame to allow movement of the additional body manipulator arm along the length of the bed frame, the body manipulator further comprising a plurality of movable elements arranged to support and move a lifting member configured to move parts of the subject anatomy to provide space for the ultrasonic probe to be moved closer to a region of the subject's anatomy and contact the region during use.
In an embodiment, the plurality of elements are interconnected by a plurality of joints and wherein each element of the additional body manipulator is rotatable relative to an adjoining element of the additional body manipulator arm.
Preferably, the base of the additional body manipulator arm is arranged to slide along the length of the bed frame to allow the additional body manipulator to be positioned in a plurality of supporting locations.
In an embodiment, the manipulator arm is mounted on a robotic trolley with at least one degree of freedom and preferably at least two degrees of freedom to undertake movement of the manipulator arm towards and away from the patient
In an embodiment, the manipulator arm comprises at least six degrees of freedom for imparting movement to the ultrasound probe module.
In an embodiment, the diagnostic imaging system further comprises a camera mounted at or adjacent the diagnostic ultrasound probe module on the robot arm to display location of the ultrasonic transducer relative to the subject's anatomy during use.
In another embodiment, the diagnostic imaging system further comprises a proximity sensor to sense distance between the diagnostic probe and the patient and wherein the proximity sensor is coupled with a proximity feedback module that communicates with the controller and the coupling arrangement to limit or effect movement of the elements of the manipulator and at least partially override the controlling input provided by the user when the distance sensed by the sensor exceeds one or more preset threshold values.
In another aspect, the invention provides method of visualising a diagnostic image, the method comprising:
Preferred features, embodiments and variations of the invention may be discerned from the following Detailed Description which provides sufficient information for those skilled in the art to perform the invention. The Detailed Description is not to be regarded as limiting the scope of the preceding Summary of the Invention in any way. The Detailed Description will make reference to a number of drawings as follows:
In a broad aspect, the invention provides a mobile, portable or fixed cardiac ultrasound imaging system 1000 which includes one or more robotic manipulator arms (as will be discussed in detail in the foregoing sections) for facilitating remote control of an ultrasound imaging system. Whilst, the present examples are limited to conducting cardiac ultrasound imaging, the scope of the invention is in no way limited to cardiac imaging only and could therefore be applied for ultrasound scanning and imaging of other anatomic parts of a subject.
Referring to
The robotic manipulator 100 comprises a fixed base with several movable parts that can move along a plurality of axes, each of the axes being located at the joints 4A to 4F with the each of the elements 5 to 11 with a diagnostic ultrasonic probe 20 being attached to lever 11. For the purposes of simplicity, each of the axes at joints 4A to 4F will be depicted by axes 1-6 in the written description. Each of the axes 1-6 is moved with a drive, for example an electric drive, which are electrically connected in a non-depicted manner to a controller 300 which includes a computer 310, so that the controller control computer 310 or a computer program running on control computer 310 is able to activate electric drives in the joints 4A to 4F in such a way that the position of element 12 to which the ultrasonic diagnostic device module 200 can be oriented essentially freely in space. The electric drives of the manipulator arm 100 each include for example an electric motor and possibly power electronics that activate the motors.
In the case of the present embodiment, these are reference coordinate systems R in the form of three-dimensional Cartesian coordinate systems each having six freedoms, particularly a world coordinate system RW of a workspace of the manipulator 100, a basic coordinate system RB in an element 5 of the manipulator 100 which forms a foot of the manipulator 100, or an ultrasonic diagnostic coordinate system RZ of the diagnostic module 200 attached to element 12 of the manipulator 100.
Movement between the plurality of elements along joints 4A to 4F is effected about each of the axes 1-6 with a drive, preferably electric drives so that the controller 300 is able to activate the electric drives in such a way so as to position or orient the ultrasonic diagnostic module 200 or its centre point freely in space.
In the case of the present exemplary embodiment, control computer 310 is programmed or designed in such a way that it, or a computer program running on it, is able to limit a work region A of the primary manipulator 100 corresponding to different operable modes. In
Choosing a specific operation limits allowable movement of the diagnostic probe module 200 within the corresponding working region that corresponds to the chosen operation mode and only allows certain pre-defined movements of the ultrasonic diagnostic module 200 thereby making operation of the primary manipulator 100 safer and preventing any inadvertent movements which might accidentally hurt the subject and also enables better image capture. Furthermore, switching between two operation modes, specifically a first and second operable mode, based on a selection on the control panel, results in the positioning of the elements of the manipulator arm 100 into the predetermined starting position for the first operable mode before effecting further movement and positioning of the elements of the manipulator to the predetermined starting position for the second operable mode thereby preventing collision between the subject's anatomy and the diagnostic probe when switching between the first and second operable modes. In many instances, the end user may commence movement of the manipulator arm 100 in the first mode to access various anatomical parts of the subject's body within the working region that corresponds to the first mode which implies that the diagnostic module 200 and the manipulator arm 100 may be positioned into one of many possible positions within the working zone that no longer corresponds to the initial predetermined starting position for the first mode. When the end user, switches from the first mode to the second mode, the controller automatically returns the manipulator arm 100 to the predetermined starting position for the first mode from the one of many possible locations within the first working zone. The system 100 prevents movement of the manipulator directly from any position within the first working region straight to the predetermined starting position of the second operable mode (which may inadvertently cause accident or hurt the subject) and forces the manipulator arm 100 to return to the predetermined starting position of the first mode before effecting further movement to the predetermined starting position of the second mode. As a result, any movements of the manipulator arm 100 and the diagnostic probe module 200 over larger ranges, in between operable modes are only carried out over safe switching paths between the operation modes which further enhances safety of the aforementioned diagnostic system 100.
During use, a subject or patient may lie on a bed frame (See
Echocardiograms may require pressing quite firmly or manipulating the ultrasound probe between ribs on the patient to obtain clear images, which mainly depends on the patient's anatomy and subcutaneous fat mass in the scanning window. The ultrasound robot pressure on the body (force) may therefore be varied depending on the anatomy of the patient. Moreover, the section of the operation window or scanning window may also limit the extent of force being applied by the manipulator robot 100. By way of example only, the force being applied by the manipulator arm 100 holding the ultrasonic diagnostic module 200 may change from 500 grams to 3 kg or more based on the operational mode being selected. Advantageously, as shown in
In addition to the force torque sensor, a proximity sensor (which may be in the form of a camera mounted at or adjacent the ultrasonic diagnostic module 200). The proximity sensor senses distance between the diagnostic probe and the patient. The proximity sensor is coupled with a proximity feedback module that communicates with the controller and the coupling arrangement to limit or effect movement of the elements of the manipulator arm 100 and at least partially override the controlling input provided by the user when the distance sensed by the sensor exceeds one or more preset threshold values. The use of the proximity sensor enables automatic changing of velocity of movement of the manipulator arm 100. As the diagnostic module gets closer to the patient's body, the approach velocity of the manipulator arm 100 may slow down in response to the increasing proximity of the diagnostic module relative to the patient's body.
Each of the predetermined starting positions for the respective operating modes may be stored in a location array that may be saved in the memory device in communication with or which is a part of the control computer 310. The stored locations may include include Cartesian coordinates (e.g., XYZ coordinates) identifying the position the various elements of the manipulator arm 100 in three dimensions. For example, the location array saved on the memory device may include the position of the particular elements which form part of the manipulator arm 100. The pre-defined range of movement for the elements comprising the manipulator arm 100 is each operable mode is typically defined by a range of allowed movements of the elements (relative to each other) and locations or coordinates within which movement of the manipulator arm 100 would be permitted when each operable mode is selected by the user. It is important to note that in conventional manipulator arms, the elements forming the manipulator arms may typically move in a plurality of different ways to position the diagnostic probe at one location in a number of different orientations. The provision of only a fixed number allowable movements for the elements of the manipulator arm 100 in combination with limiting the working zone or area for the movement of the manipulator arm 100 not only enhances safety but also aids the remotely positioned controller in avoiding incorrect orientation of the diagnostic tool avoid any undesirable or unwanted orientations when conducting an ultrasound of important regions of the patient's body such as the heart.
We refer to
In the preferred embodiment, the system also includes a user control in communication with the controller 300 over a communication network and the user control may utilise haptic feedback. In other embodiments, a controller without haptic feedback may also be used. Specifically control of the manipulator robot 100 may be achieved by using haptic feedback received on a haptic feedback controller 350. In at least some embodiments, the manipulator robot 100 may include force sensors for sensing reactive force experienced by the manipulator robot 100 during movement of the ultrasonic diagnostic module 200 as it contacts the subject's anatomy whilst operation in one of the plurality of modes. Force is a measure within each robot joint (4A to 4F), from which the end-effector force can calculated, or directly measure on the end-effector. Using the pressure (robot force) exerted by the patient body on the probe, the pressure values may be fed back to the haptic controller 350 or audio feedback device (not shown), and the operator will effectively ‘feel’ the force on the patient's body through the force (or pressure) that creates a motion in that specific direction on the haptic controller 350. The pressure experienced by the patient is displayed on a bar graph on the display device, showing the sonographer the pressure on the patient. In order to further assist the operator controlling the haptic feedback controller 350, a camera may also be mounted at or adjacent the ultrasonic diagnostic module 200 to view images of the probe as it contacts the subject's anatomy. The visual and haptic feedback combines to provide the sonographer with an indication of the pressure or torque exerted on the patient during an ultrasound scan.
The end-effector is controlled by velocity rather than position to ensure safety and accuracy. The velocity of the manipulator robot's end affecter element 12 (which is attached to the ultrasonic diagnostic module 200) is controlled by the distance the haptic controller 350 is moved. For each scanning window (corresponding a specific operational mode), a different scale factor and direction control are calibrated into the haptic feedback controller 350 to ensure safe and effective manipulator robot motion at that position. In practice, if the user moves the haptic controller 350 further, the speed at which the manipulator robot's elements move becomes faster. An operator can move the controller 350, for instance, 10 mm and hold it there, and the robot will move at a fixed velocity until the controller position is changed. A desired position of the manipulator robot 100 may also be locked to hold the ultrasound diagnostic probe module 200 at a fixed position and apply a constant force on the anatomy of the subject (a feature which is almost impossible to achieve by a sonographer manually handling the ultrasonic probe). By making a slight movement with the haptic feedback controller 350, the robot motion is nearly invisible to a human. It implies the sonographer can accurately pin-point an anatomical position by controlling the robot speed and end-effector position to millimetre accuracy.
Using the haptic controller 350, the operator can either translate or rotate the probe that forms a part of the ultrasonic diagnostic module 200. Translation speed depends on the distance the controller 350 is moved. Rotation of the probe—or ‘rocking the probe’, changes the probe angle relative to the scanning position, which is essential for changing the view window of the heart. A significant advantage of using the manipulator robot 100 to scan instead of a sonographer's hand is that the manipulator robot 100 can precisely hold the position and pressure at that point—for as long as the operator needs to capture images or do adjustments. Any movements by the patient are followed by the robot to retain the force and position on the patient.
One or more additional cameras may also be provided to allow a remotely located sonographer to view and possibly communicate with the patient relative to the manipulator arm 100. The one or more additional cameras may communicate with the controller to display captured images on a display screen being viewed by the remotely located sonographer.
The diagnostic imaging system 1000 may also include a secondary manipulator 600 shown in
The use of the diagnostic imaging system 1000 utilizes highly specialized robotic manipulators 100 and 600 in combination with an ultrasound module and a specifically programmed controller module 300 to enable improved ultrasound scanning. Typical use of the system 1000 would involve a nurse in the room directing the subject to the bed frame in a manual teleoperation mode. The sonographer would use the controller 300 to align the primary robotic manipulator 100 by activating one of the operational modes via the user input interface which would involve selection of one of the cardiac windows displayed on the display device 900. In some instances, the nurse may need to move the base of the manipulator 100 to position the manipulator robot 100 in close proximity to the bed frame before selecting one of the operational modes to position the ultrasound probe 200 and perform a scan within a scanning window corresponding to the operational mode. Once the manual robot 100's robot end-effector is positioned on one of the acoustic windows, the robotic arm's motion is constrained to a defined, limited space. A key advantage of the robot is it can hold the probe for an extended period in one position. The spatial limits are defined based on the dimension of the area scanned during a regular echocardiogram. Thus, this solution will not affect the scanning procedure. Still, it will slow the probe velocity down and prevent the sonographer from performing inadvertent movements outside of the scanning window selected, that might result in a possible collision between the robot and the patient. The sonographers may further define one or more body landmarks to further confining the scanning region for each acoustic window. A detailed flow diagram of this process has been shown in
In a typical operation, the sonographer may be located at a remote location and may connect to the robotic manipulators 100 and possibly 600 via the controller module 300 over a local LAN or WAN connection, unlock the robotic manipulators 100 and possibly 600 and request ask the nurse located in close proximity to the patient to align the patient. Once the patient has been aligned at a starting location, a scan window may be selected on the user input interface to move the primary manipulator 100 and specifically the ultrasonic module 200 mounted on the robot end effector towards the subject's anatomy. From this point, the remotely located sonographer can perform ultrasound scanning using the controller module 300. With the diagnostic probe 200 in position, the sonographer may translate the desired anatomical window (such as the heart during cardiac imaging) to find the optimum image. Once a good image is obtained, the sonographer can rock the probe by using the haptic feedback controller 350 accurately at that position to fine-tune the image.
In compliance with the statute, the invention has been described in language more or less specific to structural or methodical features. The term “comprises” and its variations, such as “comprising” and “comprised of” is used throughout in an inclusive sense and not to the exclusion of any additional features.
It is to be understood that the invention is not limited to specific features or ultrasound scanning windows shown or described since the means herein described comprises preferred forms of putting the invention into effect.
The invention is, therefore, claimed in any of its forms or modifications within the proper scope of the appended claims appropriately interpreted by those skilled in the art.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2021902680 | Aug 2021 | AU | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/AU2022/050957 | 8/23/2022 | WO |
