Patent Grant
9100307
Remote health care services, such as performing diagnostic imaging in remote locations that otherwise may not have adequate health care facilities, are increasing. The remote health care practice area is growing, due in part to cost reduction, faster diagnosis and the overall efficiency provided by a partial decentralization of health care dispensaries.
In remote health care, a patient may be examined by a remote health care practitioner (RHCP) in a medical dispensary remote from a major medical center such as a hospital. The RHCP may perform a protocol for a diagnostic test and possibly some treatment under the guidance and supervision of a specialist located at the major medical center. Communication between the RHCP and the specialist may be accomplished by a variety of communication modalities, which may exhibit communications having dynamic bandwidth, that is, a varying Quality of Service (QoS). If the effect of the varying QoS on the communications between the RHCP and the specialist is not recognized, characterized, and compensated for, in some instances the overall process is less efficient and may result in difficult or improper diagnosis.
However, in diagnostic ultrasound imaging, for example, diagnostic ultrasound image quality has no agreed upon metric. Typically, communication engineers use computational measures such as a mean squared error computed over a region of pixel values. This type of image quality assessment may be called objective. Objective quality assessment will have shortcomings for the particular computational measure selected. For example, if the mean squared error is selected as the computational error, and this error is on average very low, the assumption is that the displayed image is probably true to the original. However, a low mean squared error can arise from noise corruption that is significant in only a small portion of the transmission and this can lead to a significant image artifact. At the other end of the image quality definition are the subjective measures. These measures are typically formed by the opinion scores of expert image interpreters. Empaneling a set of experts to assess the image quality of a routine image transmission is impractical.
Thus, with conventional structures and methods, streaming data for a remote ultrasound examination does not provide a reasonable expectation of delivering a smooth presentation of adequate image quality. Moreover, even if image quality could be reliably automated so that the quality level satisfied expert image interpreters, the end goal of a transmitted image is not the quality per se, but instead the value in facilitating forming a correct diagnosis of the patient's condition. These same issues are present in conventional image compression, in particular when certifying the admissibility of lossy compression techniques and, if allowed, a determination and specification of acceptable limits on the type and degree of loss incurred by a particular compression technique. As a result, when working with transmitted medical imagery, it is typical to rely on the transmission channel's bandwidth as the main, if not sole, parameter for use in estimating image quality, which may not perform satisfactorily, particularly for a communication channel having varying bandwidth.
In one embodiment, a method for controlling transmission of medical images is provided. The method includes monitoring over time estimates of bandwidth of a channel communicatively coupling an imaging location with another location remote from the imaging location and communicating the estimates to a system at the imaging location. The method also includes identifying at least one of a region-of-interest (ROI) mask or one or more transmission parameters for the channel and adjusting at least one of the ROI mask or the one or more transmission parameters based on the monitored estimates of the bandwidth for communicating the medical images.
In another embodiment, an image communication system is provided that includes a first workstation located at one location, wherein the workstation is configured to acquire medical images from an imaging apparatus, and a transceiver coupled to the first workstation. The image communication system also includes a second workstation at a location remote from the first workstation and a transceiver coupled to the second workstation, wherein the transceivers coupled to the first and second workstations form a communication link therebetween. The image communication system further includes a link analysis unit (LAU) at the location of the second workstation, wherein the LAU is configured to monitor over time estimates of bandwidth of the communication link and communicate the estimates to the first workstation or the transceiver coupled to the first workstation. At least one of a region-of-interest (ROI) mask or one or more transmission parameters for the communication link is adjusted based on the monitored estimates of the bandwidth for communicating the medical images.
The following detailed description of certain embodiments will be better understood when read in conjunction with the appended drawings. To the extent that the figures illustrate diagrams of the functional blocks of various embodiments, the functional blocks are not necessarily indicative of the division between hardware circuitry. Thus, for example, one or more of the functional blocks (e.g., processors, controllers, circuits or memories) may be implemented in a single piece of hardware or multiple pieces of hardware. It should be understood that the various embodiments are not limited to the arrangements and instrumentality shown in the drawings.
As used herein, an element or step recited in the singular and proceeded with the word “a” or “an” should be understood as not excluding plural of said elements or steps, unless such exclusion is explicitly stated. Furthermore, references to “one embodiment” are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features. Moreover, unless explicitly stated to the contrary, embodiments “comprising” or “having” an element or a plurality of elements having a particular property may include additional such elements not having that property.
Various embodiments provide communication of imaging data, which in one embodiment is diagnostic ultrasound data. The communication channel(s) in some embodiments have a dynamic effective bandwidth and various embodiments may provide virtual streaming delivery of sequential ultrasound image frames over one or more of the communication channels having an effective bandwidth that varies. For example, typical of a protocol of diagnostic testing is an ultrasound examination. An ultrasound examination may last several minutes. It can be distracting and unhelpful to a health care professional to view the successive imagery frames if the presentation of the imagery departs from a streaming mode and there are starts and stops in the presentation. It also may be distracting and unhelpful if the image resolution degrades severely or if visual artifacts result from a poor communications channel. At least one technical effect of various embodiments is an increased stability in streaming image data, such as ultrasound image frames, wherein the streaming has reduced or eliminated stops and starts during the streaming and which may have a reduction in image resolution degradation or visual artifacts that may be caused by a communication channel having a varying effective bandwidth.
Various embodiments provide an imaging system that communicates information, such as diagnostic images, from one location (e.g., a patient examination site) to another location (e.g., a hospital remote from the examination site) over one or more communication channels, which in various embodiments, is streaming of a series or sequence of images over one or more communication channels that have a varying effective bandwidth. In one embodiment, for example, monitoring and/or continuous remote health care practitioner (RHCP) to specialist channel bandwidth estimation is provided, such as between a RHCP performing an ultrasound scan at a location remote from a health care facility and having communication channels that have varying effective bandwidths. Thus, various embodiments control the communication of a plurality of images (e.g., streaming images) over one or more communication channels.
The system 100 includes an RHCP transceiver 104 that communicates with a remote transceiver, which in the illustrated embodiment is a specialist transceiver 106. The transceivers 104, 106 communicate over or form a communication link 108, which may include one or more communication channels (e.g., cellular network communication channels), which in some embodiments have a varying effective bandwidth. Accordingly, the communication link 108 provides bi-directional or two-way communication between a first location 110 and a second location 112, which may be an examination location and a specialist location located remote therefrom (e.g., miles away), respectively, in one embodiment.
With respect to the first location 110 where the image data is acquired and processed, the RHCP workstation 102 includes a processor, which is illustrated as a computer 114. The computer 114 is coupled to the RHCP transceiver 104 to allow communication between the computer 114 and another workstation at the second location 112, illustrated as a specialist workstation 116, via the specialist transceiver 106. It should be noted that the RHCP transceiver 104 and the specialist transceiver 106 may form part of or be separate from the RHCP workstation 102 and the specialist workstation 116, respectively. It also should be noted that the workstations 102 and 116 may be any types of workstations usable by different types of operators.
The computer 114 is also connected to one or more medical devices 120 illustrated as a medical sensor suite 118. The medical devices 120 may be removably and operatively coupled to an interface (now shown) of the RHCP workstation 102 to allow communication therebetween. The medical sensor suite 118 may include a plurality of different types or kinds of medical devices, such as plurality of different types of medical imaging probes that may be used for different imaging applications. In one embodiment, the medical device 120a is an ultrasound imaging apparatus 130 as illustrated in
The computer 114 is also coupled to a user input 122 that includes one or more user controls (e.g., keyboard, mouse and/or touchpad) for interfacing or interacting with the RHCP workstation 102. The computer 114 is also coupled to a display 124, which may be configured to display one or more ultrasound images 126, such as in a time sequence or loop of images, also known as a cine loop. In operation, a user is able to control the display of the images 126 on the display 124 using the user input 122, for example, controlling the particular display settings. The user input 122 may also allow a user to control the acquisition of the image data used to generate the images 126, such as the image acquisition settings or controls. In one embodiment, the user input 122 allows control of the ultrasound imaging apparatus 130 shown in
The ultrasound imaging apparatus 130 is configured to acquire ultrasound image data that may be processed by the ultrasound imaging apparatus 130 or the RHCP workstation 102 to generate one or more images (e.g., 2D, 3D or 4D images) of a region of interest 132, for example an anatomy of interest, of a subject 134 (e.g. a patient). The ultrasound imaging apparatus 130 or the RHCP workstation 102 generates one or more images by reconstructing imaging data acquired by the ultrasound imaging apparatus 130. It should be noted that as used herein, imaging data and image data both generally refer to data that may be used to reconstruct an image.
In an exemplary embodiment, the imaging data is acquired with an imaging probe 136. In one embodiment, the imaging probe 136 may be a hand-held ultrasound imaging probe. Alternatively, the imaging probe 136 may be an infrared-optical tomography probe. The imaging probe 136 may be any suitable probe for acquiring ultrasound images in another embodiment. The imaging probe 136 is illustrated as being mechanically coupled to the ultrasound imaging apparatus 130. Alternatively or optionally, the imaging probe 136 may be in wireless communication with the ultrasound imaging apparatus 130. In still other embodiments, the imaging probe 136 is alternatively or optionally coupled to the RHCP workstation 102.
A controller 138 is provided to control scan parameters of the imaging probe 136. For example, the controller 138 may control acquisition parameters (e.g. mode of operation) of the imaging probe 136. In another embodiment, the controller 138 may control other scan parameters (e.g. gain, frequency, etc.) of the imaging probe 136. The controller 138 may control the imaging probe 136 based on scan parameters provided by an operator at a user interface 140. The operator may set the scan parameters of the imaging probe prior to image acquisition with the imaging probe 136. In one embodiment, the operator may adjust the scan parameters of the imaging probe during image acquisition. It should be noted that in some embodiments, the controller 138 is embodied as the computer 114 (shown in
An imaging module 142 is provided to reconstruct one or more images based on the imaging data acquired by the imaging probe 136. The imaging module 142 may include a processor or computer that reconstructs the images, which in some embodiments, forms part of the computer 114 (shown in
In one embodiment, a display 144 (which may be embodied as the display 124 of
In operation, data acquired by the ultrasound imaging apparatus 130 is accessible and may be communicated between the first location 110 and the second location 112 using the transceivers 104, 106. It should be noted that the transceivers 104, 106 may be configured to communicate using any suitable communication protocol, such as a suitable wireless communication protocol, such as cellular 3G communication protocols. Using this arrangement, data from the computer 114 at the RHCP workstation 102 may be transmitted to a specialist at the specialist workstation 116 and data sent from the specialist may be received at the RHCP workstation 102.
At the second location 112, which in one embodiment may be a hospital or health care facility having a specialist located there, a link analysis unit (LAU) 150 is configured to monitor the data received from the first location 110, via the transceivers 104, 106 and provide data to the specialist workstation 116 and/or the specialist transceiver 106 as described in more detail herein.
In various embodiments, the LAU 150 is a module or controller, which may be implemented in hardware, software, or a combination thereof. The LAU 150 is located proximate the specialist workstation 116, which in some embodiments forms part of the specialist workstation 116 or may be a module operatively coupled to the specialist workstation 116. The specialist workstation 116 may be a data server where multiple workstations may be connected and interacting with the RHCP workstation 102. The LAU 150 is configured to perform one or more functions, which in the illustrated embodiment allows for a bandwidth monitoring or continuous RHCP to specialist channel bandwidth estimation, which is communicated to the RHCP transceiver 104 via the specialist transceiver 106.
In particular, the LAU 150 is configured to provide an estimate of the channel bandwidth (e.g., bandwidth of the communication link 108) with a confidence interval based on a channel model. In various embodiments, the estimated value is used with a rate-distortion model to predict the image quality during T seconds of the examination, to support a virtual streaming display (such as a smoothly presented display of ultrasound images over time without time breaks) of an ultrasound examination of length T seconds without decreasing image quality below a specified level. As illustrated in time graph 160 of
In particular, in one embodiment, initialization of the workstations 102, 116 prior to an ultrasound examination proceeds as follows (with reference to
1. The operator enters the duration, T, of the ultrasound examination's imagery to be communicated across the RHCP to specialist communications link 108 and smoothly displayed to the specialist (e.g., to allow diagnostic analysis thereof), the initial ROI contraction mask 170 (shown in
2. The two-way communication link 108 between the RHCP workstation 102 and the specialist workstation 116 is established and channel monitoring and modeling initiated and continued for a duration of at least TS.
3. The LAU 150 determines if the link 108 can be reasonably expected to provide a smooth streaming presentation of adequate image quality additionally delayed by no more than ΔT between the display 124 of the RHCP workstation 102 and the display 117 of the specialist workstation 116. The analysis may include any suitable means for channel bandwidth monitoring.
4. If the determination in step 3 is positive, the examination proceeds. If the determination in step is not positive, the ΔT period is adjusted based on the minimum image quality and estimated channel bandwidth or, for example, the system can switch to a “store and forward” mode where data is stored or buffered before sending.
Once the examination is in process, the RHCP or the specialist may adjust the ROI contraction mask and the position of the mask's center as described in more detail herein.
In various embodiments, the LAU 150 is operable to control the transmission of information over the communication link 108 using one or more different methods. For example, in various embodiments, the LAU 150 is operable to control a region-of-interest (ROI) contraction mask and the degree of lossy compression, which will be described in more detail herein. Both of these parameters may be used to lower the transmission rate from the RHCP workstation 102 while still ensuring image quality. In particular, the region of interest (ROI) contraction mask 170 as shown in
In some embodiments, the ROI contraction mask 170 may be contracted in an area 174 as the available bandwidth drops on the communication link 108 from the RHCP to the specialist. It should be noted that as used herein, contraction of the ROI contraction mask 170 (also referred to as the ROI mask 170) generally means that the ROI mask 170 is decreased or shrunk in physical size or extent, which may be proportionally applied along the entire boundary defined by the ROI mask 170. Alternatively, or in conjunction with the contraction of the ROI mask 170, the lossy compression may be increased as the available bandwidth decreases, illustrated by the fuzzy lines 172a. In this alternative embodiment, it should be noted that the increase in lossy compression may be only outside the image area enclosed by the ROI mask 170, only inside the image area enclosed by the ROI mask 170, or both outside and inside the image area enclosed by the ROI mask 170. For example, a compression rate outside of the ROI mask 170 may be reduced to maintain a constant compression rate inside the ROI mask 170, which include the image data of particular interest. Other parameters for communicating data associated with the image portion inside and outside of the ROI mask 170 also may be adjusted and may be different for each.
Contraction of the ROI mask 170 may be performed, for example, by a window analysis module (WAM) 125 operating on the computer 114 of the RHCP workstation 102. In one embodiment, the WAM module 125 may be programmed to perform the ROI mask contraction according to one or more of the following:
1. An RHCP's or specialist's stored preferred procedure;
2. A recommended or specified procedure;
3. A real-time occasional or continuous adjustment by the RHCP or the specialist; and/or
4. On command by the LAU 150 if the available bandwidth drops below the level needed to sustain the required or desired transmission rate.
With respect to control by the LAU 150 (4. above), and as an example, assume that the ROI is a circular region of radius R and that the area of a pixel is A. The ROI thus comprises
pixels. Further, assume that coding assigns B bits per pixel on the average. The amount of ROI image data, D, associated with the ROI is then
bits. The two variables that the LAU 150 may use to fulfill control of the data transmission are R and B. The sensitivities of the amount of ROI image data to these variables may be defined as follows:
Because R is more nearly a continuous variable than B, it should be noted that a reduction in the amount of ROI image data may be effected by reducing the average number of bits per pixel rather than reducing B by an integer amount. In one embodiment, the WAM 125 may be programmed to change D according to a directional derivative such as α(R,B)DR+β(R,B)DB where the coefficients {α(R,B),β(R.B)} are provided by a table or function.
In one embodiment, the feedback estimate of the RHCP to specialist channel feedback that controls the WAM 125 may also facilitate notification by the RHCP by causing an acoustic signal such as a tone to be emitted from the workstation, the frequency of the tone indicative of the channel's condition. For example, a relatively high audio frequency might indicate good channel quality and a relatively low frequency might indicate a degraded channel quality. Also, visible indicators, such as a changing light may be provided.
It should be noted that the use of lossy coding and progressive transmission may be used if and as the link quality degrades. The choice of a lossy coding technique is used in various embodiments to maintain adequate image quality. For example, sub-band coding used with progressive transmission reduces or avoids block artifacts that may arise when using compression methods based on block transforms such as with some JPEG compression. The JPEG2000 compression technique is based on wavelets and is well suited to progressive transmission resulting in image quality improvement as more data is received. It should be noted that JPEG is merely one example of a compression technique and various embodiments are not limited to a particular compression. For example, other types of compression includes, MPEG and h.264, among others.
In another embodiment, the lossy compression method selected for use with progressive transmission may involve some despeckling of the ultrasound images. Narrowband ultrasound is a coherent imaging technique and the images produced thereby are suffused with speckle noise. It should be noted that speckle noise behaves as multiplicative noise rather than additive. Accordingly, the speckle phenomenon may be defined in a quasi-linear fashion by considering the logarithm of the image such as in performing a multi-resolution decomposition. Nonlinear processing methods also may be used, such as applying a bank of oriented one-dimensional median filters to the image and setting each point's value to the largest value reported from all of the filters in the filter bank. This method reduces speckle noise and preserves the image's structure and in particular the “thin bright streaks” which occur between tissue layers when imaging tissue. Based on wavelet transforms, another nonlinear technique that may be used is described in U.S. Pat. Nos. 5,619,998 and 5,497,777, both of which are commonly assigned.
The LAU 150 also may estimate the QoS of the received symbols (e.g., data bits) from the RHCP transceiver 104, which is described in more detail herein. This estimation may be used to perform improved decoding of some forward error correction codes.
The LAU 150 may also create a banked or stored bandwidth-time product. This is achieved by delaying the imagery display to the specialist for a period of ΔT seconds. The time interval ΔT is an effective bank or buffer to be used if the RHCP to specialist channel (e.g., the communication link 108) becomes momentarily unusable.
Various embodiments provide for controlling the transmission of image data via a communication link, particularly a communication link with a changing effective bandwidth. In particular, as described herein, a combination of signal processing and a control loop arrangement may be used to control, for example, ultrasound imaging over one or more channels having a dynamic effective bandwidth. As described herein, various embodiments may monitor, which can include continuously estimating an RHCP to specialist channel bandwidth, which is then forwarded to the RHCP transceiver 104 or computer 114 and used as feedback to control the transmission of the image data. Additionally, an RHCP or the defined ROI mask 170 may be applied to the RHCP imager and compression and/or progressive transmission of the ultrasound imagery may be performed. Also, as described herein, a selectable time interval may be used that extends the imagery sequence transmission time by ΔT.
Using various embodiments, the monitoring (which in some embodiments includes the continuous estimating) of the QoS of the communication link 108 is performed. Thus, by dynamically evaluating the data channel, data flow may be controlled. For example, a method 200 for controlling image data communication is shown in
The method 200 also includes communicating the estimates to the system at the imaging location at 204, for example, to control a transceiver at the imaging location as described in more detail herein. This estimate information provides a feedback loop that may be used to control the transmission of the image data over the channel.
The method 200 also includes identifying an ROI mask and/or transmission parameters at 206. For example, as described herein the ROI mask 270 may be selected by a user. In other embodiments, the ROI mask (e.g., size of the ROI mask) and/or the transmission parameters, for example, parameters for compression and progressive transmission of the image data are initially selected based on the estimates.
The method 200 also includes adjusting the ROI and/or the transmission parameters based on the estimates. For example, a compression rate for the data may be changed, which may include changing the compression rate inside and/or outside of the ROI mask. In some embodiments, the compression rates inside and outside are changed or adjusted differently, which in various embodiments includes adjusting the compression rate outside of the ROI before adjusting the compression rate inside the ROI.
In various embodiments, motion information may be used to define the ROI mask. For example, areas of motion within the series of images may be used to select a region for the ROI mask, which then defines how much data is communicated. For example, more data may be communicated corresponding to the area inside the ROI mask than outside the mask because the ROI mask includes therein moving regions that are of particular interest. It should be noted that the ROI mask may change shape, such as based on a user input or available bandwidth estimates. It also should be noted that while different regions are defined inside and outside the ROI mask, for example, the ROI mask 170 shown in
Thus, the ROI mask may be defined by the channel measurements to selected compression (e.g., rates or types) and/or frame rates inside and outside the ROI mask. In general, the compression and/or frame rate is defined such that more image data is communicated for the area inside the ROI mask than outside the ROI mask. It should be noted that the change in compression rates of having different compression may include using a different compression ratio, or in some embodiments, using a different type of compression.
It should be noted that a selectable time interval may be provided as described in more detail herein. For example, bandwidth may be accumulated or held to allow for a more continuous data stream, such as if there is an acceptable amount of delay.
It also should be noted that in some embodiments, the ROI mask may be automatically selected and initial compression and frame rate parameters set. For example, moving regions of the image (e.g., motion of heart valves) may be included in the ROI mask. Additionally, in some embodiments, sub-regions within the ROI mask or outside the ROI mask may be identified and include different compression or frame rates. For example, in color flow imaging different colors within the image may change at different rates. This can occur, for example, when venous flow and arterial flow are both visible, since venous flow tends to be continuous while arterial flow is pulsatile at the heart rate. In some embodiments, different colored regions having different rates of changes in color or intensity are used to change or adjust (or initially set) the compression and/or frame rates. In other embodiments, using known landmarks, areas of more blood flow may be predicted and included in the ROI mask, which may also be used to set or change the compression and/or frame rates.
It should be noted that various embodiments provide for increasing the likelihood that information of more clinical relevance (e.g., moving portions of the images) are communicated and displayed to the user remote from the imaging site in a manner that allows viewing and analysis thereof (e.g., a smoothly displayed series of images without jumping or starts/stops). In some embodiments, the compression scheme and/or frame rate may be changed based on a particular imaging preset or protocol, such as based on a particular scan to be performed.
In some embodiments, the frame rate may be reduced. For example, in various embodiments, ultrasound data may be acquired at 100 frames/second. These higher frame rates may be used for particular types of imaging, such as imaging the fetal heart. In these cases, the frame rate is typically not reduced, although the part of the ROI that is of particular interest may be very small. However, in other cases, for example, when the frame rate reaches 40 or 45 frames per second, such as because of the imaging depth or sector width, the frame rate may be reduced or limited since in general, this may not be clinically relevant or important. For example, in such cases, the data may be slowed, such as to 30 frames/second or other value, which may be a predetermined or defined value. Thus, in some embodiments, a determination is first made as to whether it is acceptable to reduce the frame rate (e.g., based on the type of imaging) before the frame rate is reduced.
The various embodiments may be implemented in connection with different imaging systems, such as different ultrasound imaging systems. For example,
The probe 306 may be coupled to the system 300 with wires, cable, or the like. Alternatively, the probe 306 may be physically or mechanically disconnected from the system 300. The probe 306 may wirelessly transmit acquired ultrasound data to the system 300 directly or through an access point device (not shown), such as an antenna disposed within the system 300.
The user interface 356 also includes control buttons 358 that may be used to control the system 350 as desired or needed, and/or as typically provided. The user interface 356 provides multiple interface options that the user may physically manipulate to interact with ultrasound data and other data that may be displayed, as well as to input information and set and change scanning parameters and viewing angles, etc. For example, a keyboard 360, trackball 362, and/or other controls 364 may be provided. One or more probes (such as the probe 136 shown in
One or more embodiments may provide real-time transmission of data to enable clinically viable examination and diagnosis from different locations.
The various embodiments and/or components, for example, the modules, or components and controllers therein, also may be implemented as part of one or more computers or processors. The computer or processor may include a computing device, an input device, a display unit and an interface, for example, for accessing the Internet. The computer or processor may include a microprocessor. The microprocessor may be connected to a communication bus. The computer or processor may also include a memory. The memory may include Random Access Memory (RAM) and Read Only Memory (ROM). The computer or processor further may include a storage device, which may be a hard disk drive or a removable storage drive such as a floppy disk drive, optical disk drive, flash drive, jump drive, USB drive and the like. The storage device may also be other similar means for loading computer programs or other instructions into the computer or processor.
As used herein, the term “computer” or “module” may include any processor-based or microprocessor-based system including systems using microcontrollers, reduced instruction set computers (RISC), application specific integrated circuits (ASICs), logic circuits, and any other circuit or processor capable of executing the functions described herein. The above examples are exemplary only, and are thus not intended to limit in any way the definition and/or meaning of the term “computer”.
The computer or processor executes a set of instructions that are stored in one or more storage elements, in order to process input data. The storage elements may also store data or other information as desired or needed. The storage element may be in the form of an information source or a physical memory element within a processing machine.
The set of instructions may include various commands that instruct the computer or processor as a processing machine to perform specific operations such as the methods and processes of the various embodiments. The set of instructions may be in the form of a software program. The software may be in various forms such as system software or application software and which may be embodied as a tangible and non-transitory computer readable medium. Further, the software may be in the form of a collection of separate programs or modules, a program module within a larger program or a portion of a program module. The software also may include modular programming in the form of object-oriented programming. The processing of input data by the processing machine may be in response to operator commands, or in response to results of previous processing, or in response to a request made by another processing machine.
As used herein, the terms “software” and “firmware” are interchangeable, and include any computer program stored in memory for execution by a computer, including RAM memory, ROM memory, EPROM memory, EEPROM memory, and non-volatile RAM (NVRAM) memory. The above memory types are exemplary only, and are thus not limiting as to the types of memory usable for storage of a computer program.
It is to be understood that the above description is intended to be illustrative, and not restrictive. For example, the above-described embodiments (and/or aspects thereof) may be used in combination with each other. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the various embodiments of the described subject matter without departing from their scope. While the dimensions and types of materials described herein are intended to define the parameters of the various embodiments, the embodiments are by no means limiting and are exemplary embodiments. Many other embodiments will be apparent to one of ordinary skill in the art upon reviewing the above description. The scope of the various embodiments should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.” Moreover, in the following claims, the terms “first,” “second,” and “third,” etc. are used merely as labels, and are not intended to impose numerical requirements on their objects. Further, the limitations of the following claims are not written in means-plus-function format and are not intended to be interpreted based on 35 U.S.C. §112, sixth paragraph, unless and until such claim limitations expressly use the phrase “means for” followed by a statement of function void of further structure.
This written description uses examples to disclose the various embodiments, including the best mode, and also to enable one of ordinary skill in the art to practice the various embodiments, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the various embodiments is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if the examples have structural elements that do not differ from the literal language of the claims, or if the examples include equivalent structural elements with insubstantial differences from the literal languages of the claims.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5497777 | Abdel-Malek et al. | Mar 1996 | A |
| 5619998 | Abdel-Malek et al. | Apr 1997 | A |
| 5877801 | Martin et al. | Mar 1999 | A |
| 6045507 | Muzilla et al. | Apr 2000 | A |
| 6633674 | Barnes et al. | Oct 2003 | B1 |
| 6829391 | Comaniciu et al. | Dec 2004 | B2 |
| 7076091 | Rosenfeld | Jul 2006 | B2 |
| 8031771 | Chen et al. | Oct 2011 | B2 |
| 8184069 | Rhodes | May 2012 | B1 |
| 8868716 | Muralidharan | Oct 2014 | B2 |
| 20040002652 | Phelps et al. | Jan 2004 | A1 |
| 20040143679 | Ogikubo | Jul 2004 | A1 |
| 20050058197 | Lu et al. | Mar 2005 | A1 |
| 20060058609 | Olstad | Mar 2006 | A1 |
| 20060122482 | Mariotti et al. | Jun 2006 | A1 |
| 20070113242 | Fetkovich | May 2007 | A1 |
| 20070237402 | Dekel et al. | Oct 2007 | A1 |
| 20080077702 | Posamentier | Mar 2008 | A1 |
| 20080218588 | Stetten | Sep 2008 | A1 |
| 20100080476 | Khorasani et al. | Apr 2010 | A1 |
| 20100202510 | Kyle | Aug 2010 | A1 |
| 20100298711 | Pedersen et al. | Nov 2010 | A1 |
| 20120327218 | Baker et al. | Dec 2012 | A1 |
| Number | Date | Country |
|---|---|---|
| 101904751 | Dec 2010 | CN |
| 102327131 | Jan 2012 | CN |
| 2008066682 | Jun 2008 | WO |
| Entry |
|---|
| De Cunha, D et al., “The MIDSTEP system for ultrasound guided remote telesurgery”, Engineering in Medicine and Biology Society, 1998. Proceedings of the 20th Annual International Conference of the IEEE, Issue Date: Oct. 29-Nov. 1, 1998, on page(s): 1266, Print ISBN: 0-7803-5164-9. |
| Unofficial translation of Chinese Office Action and Search Report from corresponding CN Application No. 201310674509.X dated Mar. 19, 2015. |
| Number | Date | Country | |
|---|---|---|---|
| 20140164596 A1 | Jun 2014 | US |
