The following relates generally to the imaging arts, imaging device setting modification arts, remote imaging monitoring arts, and related arts.
The advantages of having a local technologist assisted by a more experience remote technologist or radiologist has been recognized. A suitable approach uses a screen sharing application in which the imaging device controller's display is mirrored at a remote display, along with a video camera installed in the imaging bay containing the medical imaging device, which provides a remote video feed showing the patient and capturing patient/operator interactions, and a telephone link or other natural language communication link via which a remote “operator” can advise the local operator.
Due to safety considerations, the mirrored remote display is “read-only” during clinical imaging, and the remote “operator” is unable to actually control the medical imaging device. Rather, the remote “operator” provides advice via the telephone link to a local operator, who then actually operates the medical imaging device. Consequently, the remote assistance process is cumbersome, as the remote party must make suggestions to the local technologist via telephone, instant messaging, or the like, and the suggestions are actually input to the imaging device controller by the local technologist.
The following discloses certain improvements to overcome these problems and others.
In one aspect, a control system for controlling a medical imaging device includes: an electronic controller including controller memory and a controller display. The electronic controller is operatively connected to control the medical imaging device and programmed to: store a controller state in the controller memory; execute imaging device setup software to cause the electronic controller to provide a setup user interface on the electronic controller via which a user configures the controller state stored in the controller memory; and execute control software to cause the electronic controller to control the medical imaging device to acquire images in accordance with the controller state stored in the controller memory. A remote computer includes remote computer memory and a remote computer display. The remote computer not operatively connected to control the medical imaging device and programmed to: receive the controller state from the electronic controller via an electronic network and store the received controller state in the remote computer memory, execute the imaging device setup software to cause the remote computer to provide the setup user interface on the remote computer via which a remote user configures the controller state stored in the remote computer memory thereby creating controller state modifications, and transfer the controller state modifications to the electronic controller. The electronic controller is further programmed to update the controller state stored in the controller memory with the controller state modifications.
In another aspect, a non-transitory storage medium stores instructions readable and executable by a remote computer including remote computer memory and a remote computer display to perform a remote assistance method for assisting a user of an electronic controller connected to control a medical imaging device wherein the electronic controller is programmed to execute imaging device setup software. The method includes: receiving a controller state of the electronic controller via an electronic network and storing the received controller state in the remote computer memory; executing a copy of the imaging device setup software of the electronic controller to cause the remote computer to provide a setup user interface on the remote computer via which a remote user configures the controller state stored in the remote computer memory thereby creating controller state modifications, and transferring the controller state modifications from the remote computer to the electronic controller.
In another aspect, a control method for controlling a medical imaging device using an electronic controller includes: storing a controller state of the medical imaging device in a controller memory of the electronic controller; controlling the medical imaging device using the electronic controller to acquire images accordance with the controller state stored in the controller memory; providing a setup user interface on the electronic controller via which a user configures the controller state stored in the controller memory, wherein the setup user interface includes a user input operable by the user to obtain remote setup assistance by operations performed by the electronic controller including: transferring the controller state to a remote computer via an electronic network; after the transferring, receiving controller state modifications from the remote computer via the electronic network; and updating the controller state stored in the controller memory with the controller state modifications.
One advantage resides in providing an ability to remotely modify one or more settings of an imaging device, without introducing patient safety risks associated with remote control of a medical imaging device.
Another advantage resides in providing a virtual twin of an imaging device that is remote from the imaging device that provides for remote modification of one or more settings of the imaging device.
Another advantage resides in providing a virtual twin of an imaging device that allow modifications to settings of the imaging device without actually implementing the modifications.
A given embodiment may provide none, one, two, more, or all of the foregoing advantages, and/or may provide other advantages as will become apparent to one of ordinary skill in the art upon reading and understanding the present disclosure.
The disclosure may take form in various components and arrangements of components, and in various steps and arrangements of steps. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the disclosure.
In some embodiments disclosed herein, a twin of the imaging device controller is provided at the remote computer. The twin comprises a remote computer, or a virtual machine (VM) installed on a remote computer, which runs the same operating system (OS) and software as that running on the imaging device controller. The remote computer or VM also has loaded therein an image copy of all data currently stored in the memory of the imaging device controller. This includes the current hardware configuration files, current software-adjustable hardware settings, current imaging sequence settings, and any acquired images currently stored at the controller.
The twin running on the remote computer or VM is typically not intended to allow for remote control of the imaging device; accordingly, the twin does not need to include simulation of input/output interactions with the hardware of the imaging device. However, the hardware configuration files and software-adjustable hardware settings are transferred to the twin because the available hardware and their settings can impact permissible ranges of values for the imaging sequence settings. Providing any acquired images at the twin also allows the remote party to review the images to assess image quality, including performing any type of image processing that could be performed at the imaging device controller.
Hence, the remote party can interact with the twin as if interacting with the actual imaging device controller. This allows the remote party to efficiently adjust combinations of imaging sequence settings and/or software-adjustable hardware settings based on the knowledge and experience of the remote party. As the remote party is typically an experienced imaging technician or other domain expert having strong familiarity with the make/model of the imaging device, the twin provides the remote party with (a copy of) the actual imaging device controller user interface, with which the remote party is already comfortable.
To address safety concerns with remote operation, the remote party cannot actually apply any changes made to the twin running on the remote computer or VM to the actual imaging device controller. Rather, the twin maintains a record of all adjustments (e.g. hardware setting adjustments, imaging sequence setting changes) applied to the twin by the remote party. When the remote party is satisfied with the adjustments, the remote party pushes the adjustments to the imaging device controller. At the controller, these adjustments to the twin are presented to the local technician as proposed changes. The local technician can then accept or reject each proposed adjustment (or, in a variant approach, can accept or reject groups of related adjustments, say to a specific imaging sequence, or can accept or reject all adjustments) and the accepted adjustments are applied to the imaging device controller. An indication of any rejection is preferably sent back to the twin and the rejected adjustment(s) is/are rolled back at the remote computer or VM.
If further remote assistance is desired (e.g., if the initial set of adjustments does not solve the problem that led the local technician to seek the assistance of the remote party), then the twin running on the remote computer or VM is updated to the current state of the controller (e.g., memory contents copied to the twin along with any additional adjustments made by the local technician beyond those proposed adjustments that were accepted) and the process is repeated.
Because the twin is not performing remote control of the imaging device, the twin is not performing screen-sharing (read-only or otherwise) of the controller display at the twin. Likewise, the display of the twin is not shared at the local controller.
In some embodiments disclosed herein, read-only screen sharing of the display of the twin is provided at the local controller, e.g. in a separate window shown on the controller display using a conventional screen-sharing application. This could be educational for the local technician who can thereby follow the (proposed) adjustments step-by-step as they are performed by the remote party. The local technician might also provide real-time feedback via telephone or instant messaging based on the shared screen (for example, “I already tried that adjustment and it did not improve the image.”). However, any such screen sharing of the twin display at the controller would be read-only and would not actually change any settings of the imaging device controller. In similar fashion, it is contemplated to provide screen-sharing of the display of the local controller at the remote service center that hosts the remote party. This could be useful to allow the remote party to observe the local technician's manipulation of the local controller in real time. Again, however, any such screen sharing of the local controller at the remote service center would be read-only and additional to the display of the twin presented to the remote party.
With reference to
The controller memory 26 is programmed to store hardware and software related to operation of the imaging device 12. For example, the controller memory 26 is programmed to store a controller state 30 of the imaging device 12. The controller state 30 can include operational settings of the imaging device 12. The controller settings 32 can include image acquisition settings 32. In another example, the controller state 30 includes a device hardware configuration 34 for the medical imaging device 12. In a further example, the controller state 30 includes images 36 stored in the controller memory 26.
The controller memory 26 stores imaging device setup software 38 that allows a user (e.g., a radiologist, etc.) to configure the settings 32 of the controller state 30. The controller memory 26 also stores control software 40 that causes the electronic controller 18 to control the medical imaging device 12 to acquire the images 36 of a patient in accordance with the stored controller state 30. Usually, the controller memory 26 further stores image presentation software 42 to cause the electronic controller 18 to display and manipulate the images 36 on the controller display device 24. Furthermore, as disclosed herein, the controller memory 26 stores remote update software 44 that causes the electronic controller 18 to update the controller state 30 based on feedback received from the twin, as will be described herein. In addition, the electronic controller 18 is programmed to run an operating system (OS) 48 under which the imaging device setup software 38, the control software 40, the image presentation software 42, and the remote update software 44 are executed.
The electronic controller 18 is configured as described above to perform a control method or process 100 for controlling the medical imaging device 12 via the control software 40 to acquire medical images of patients in accordance with the settings configured via the setup software 38, and for presenting the acquired images for review by the technician via the presentation software 42, and as disclosed herein for receiving remote updates from the twin via the remote update software 44. The non-transitory storage medium 26 stores the instructions 38, 40, 42, 44 which are readable and executable by the at least one electronic processor 20 of the electronic controller 18 to perform disclosed operations including performing the control method or process 100. In some examples, the method 100 may be performed at least in part by cloud processing.
With reference to
At an operation 104, the electronic controller 18 is programmed to execute the imaging device setup software 38 to cause the electronic controller to provide the setup user interface 28 on the GUI to allow a user to configure the settings 32 of the controller state 30. The setup user interface 28 is configured in accordance with the device hardware configuration 34 of the controller state 30.
At an operation 106, the electronic controller 18 is programmed to execute the control software 40 to cause the electronic controller to control the medical imaging device 12 to acquire the images 36 in accordance with the stored controller state 30. The images 36 can be stored in the controller memory 26. Additionally or alternatively, the electronic controller 18 is programmed to execute the image presentation software 42 to display and manipulate the images 36 on the display device 24. The manipulated images 36 can also be stored in the controller memory 26. The process 100 may stop at 106 if the local technician is satisfied with the images and the imaging examination is complete; otherwise, the process passes back to operation 104 where the local operator adjusts the settings to improve the images, or moves on to another set up another imaging sequence to be performed in the imaging examination.
Referring back to
The remote computer 18′ is configured as an electronic processing device, a workstation computer, or more generally a computer, and is not operatively connected to control the medical imaging device 12. Alternatively, the remote computer 18′ can be embodied as a server computer or a plurality of server computers, e.g. interconnected to form a server cluster, cloud computing resource, or so forth. The electronic controller 18 includes typical components, such as an electronic processor 20′, at least one user input device 22′, and a remote computer display device 24′. In some embodiments, the display device 24′ can be a separate component from the remote computer 18′. The electronic processor 20′ is operatively connected with a remote computer memory 26′ embodied as one or more non-transitory storage media. The remote computer non-transitory storage media 26′ stores instructions executable by the at least one electronic processor 20′. The instructions include instructions to generate a setup GUI 28′ for display on the remote computer display device 24′.
The remote computer 18 implements the twin directly on the remote computer 18′, or in a VM running on the computer 18′. Direct implementation may be feasible if the remote computer 18′ is running the same operating system as the electronic controller 18. However, implementation of the twin in a VM running on the remote computer 18′ has a number of advantages. It permits the remote computer 18′ to be running a different operating system (OS) than the OS 48 of the electronic controller 18. Furthermore, using a VM places the twin in a sandbox, meaning that the twin running in a VM does not impact other processes that may optionally be running on the remote computer 18′. Such “other processes” may, for example, include an electronic mail program, word processor, or other programs that may be used by the remote party, and/or may include video streaming software for receiving and presenting video of the imaging device 12 acquired by a video camera (not shown) in the imaging bay, and/or may include screen sharing software providing the remote party with a real-time view of content shown on the display 24 of the electronic controller 18. Yet a further advantage to using a VM is that if the remote party is providing assistance to multiple different local technicians in different imaging bays, then the remote computer 18′ can run multiple twins (one for each of the different imaging bays) in separate VMs, and again the use of VMs ensures that the twins do not interfere with each other. Some suitable VM environments are available, for example, from VMware, Inc. (Palo Alto, Calif., USA) or Parallels Inc. (Bellevue, Wash., USA), by way of non-limiting illustrative example.
The remote computer memory 26′ also stores a copy 30′ of the controller state 30, (including a copy 32′ of the controller settings 32, a copy 34′ of the device hardware configuration 34, and a copy 36′ of the images 36), a copy 38′ of the imaging device setup software 38, a copy 42′ of the image presentation software 42, and a copy 48′ (i.e. second instance 48′) of the OS 48. Typically, the remote computer memory 26′ does not store a copy of the control software 40, since the remote computer 18′ is not intended to control the imaging device 12. However, in some specific implementations the imaging device setup software 38 and the control software 40 may be integral with one another, i.e. written as a single piece of software, or the two pieces of software 38, 40 may only available and installable as a combined package. In this case, a copy of the combined software 38, 40 is loaded onto the remote computer 18′ (or the VM running on same) in order to provide the copy 38′ of the device hardware configuration 34, and the control imaging device control aspects of this combined software package will be inoperable. The OS 48 runs on the remote computer 18′ or on a VM executing on the remote computer 18′, and allows the user to execute (the copy of) the image device setup software 38′ under the (copy or second instance of) OS 48′ running on the remote computer 18′ or VM, and allows the user to execute (the copy of) the image presentation software 42′ under the (copy or second instance of) OS 48′ running on the remote computer 18′ or VM.
Furthermore, the remote computer memory 26′ stores modification tracking software 54 which, when executed by the remote computer 18′ or VM running on same operates to track controller state modifications made on the twin and ultimately to push the modifications to the electronic controller 18. The modification tracking software 54 may execute as a background process that monitors and tracks changes made to (the copy of) the controller state 30′. Alternatively, if the imaging device setup software 38′ has “undo” capability then the imaging device setup software 38′ will be logging modifications automatically (in order to enable undoing them if the operator so requests), and in this case the modification tracking software 54 may operate by referencing the “undo” stack of modifications automatically maintained to facilitate the “undo” capability. In order to receive the controller state 30 from the electronic controller 18 and to push the state modifications made on the twin to the electronic controller 18, the remote computer 18′ is operatively connected to the electronic controller 18 via an electronic network 46 (e.g., Internet, Wi-Fi, a personal area network (PAN), a virtual private network (VPN), and so forth). Optionally, the electronic network 46 may provide other connectivity between the imaging bay of the imaging device 12 and the remote servicing center hosting the remote computer 18′, such as video streaming capability for streaming video of the imaging device 12 to the remote computer 18′, transferring screen sharing data (this screen sharing again would be separate from operation of the twin described herein), and providing a natural language link such as a video chat or voice over Internet protocol (VOIP) telephonic link via which the local technician and the remote operator can converse.
The remote computer 18′ is configured as described above to perform a remote assistance method or process 200 for assisting a user of the electronic controller 18 connected to control the medical imaging device 12. The non-transitory storage medium 26′ stores instructions which are readable and executable by the at least one electronic processor 20′ of the remote computer 18′ to perform disclosed operations including performing the remote assistance method or process 200. In some examples, the method 200 may be performed at least in part by cloud processing.
With reference to
At an operation 204, the remote computer 18′ is programmed to execute the imaging device setup software 38′ to provide the setup user interface 28′ on the remote computer display device 24′ via which a remote user (e.g., a technologist) configures the received controller state 30′ to create controller state modifications 30m. The setup user interface 28′ is configured in accordance with the device hardware configuration 34′. The controller state modifications 30m typically include modifications to the image acquisition settings 32′.
At an operation 206, the remote computer 18′ is programmed to execute the image presentation software 42′ to display and manipulate the images 36 stored in the remote computer memory 26′ on the remote computer display device 24′. The manipulated images 36 can also be stored in the remote computer memory 26′ and can constitute controller state modifications 30m. (Depending upon the nature of the assistance being provided, the remote party may omit the operation 206, i.e. may not load and use the image presentation software 42′).
At an operation 208, the remote computer 18′ is programmed (e.g., by the modification tracking software 54) to transfer the controller state modifications 30m to the electronic controller 18. At this point, the control method 100 resumes. It will be appreciated that the transferred control state modifications 30m are not automatically implemented into the controller state 30 without authorization from the radiologist at the imaging device 12. In other words, the remote computer 18′ cannot control the imaging device 12.
Rather, referring back to
In some examples of some suitable implementations of the update operation 108 and the update GUI 50, the controller state modifications 30m are displayed as a list on the controller display device 24 in which the radiologist can accept or reject individual ones of the controller state modifications (e.g., via the at least one user input device 22). For those controller state modifications 30m that are accepted, the accepted modifications are implemented into the controller state 30. For the rejected controller state modifications 30m, a message is sent to the technologist that is displayed on the remote computer display device 24′ indicating which modifications were not accepted by the local technologist, and the rejected modifications are optionally rolled back at the remote computer 18′. The operations 204 and/or 206 can be repeated to generate a new set of modifications, which are then transmitted to the electronic controller 18, until the local technologist accepts all of the modifications.
The disclosure has been described with reference to the preferred embodiments. Modifications and alterations may occur to others upon reading and understanding the preceding detailed description. It is intended that the exemplary embodiment be construed as including all such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/EP2020/081789 | 11/11/2020 | WO |
Number | Date | Country | |
---|---|---|---|
62934027 | Nov 2019 | US |