Patent Application
20140323856
This disclosure relates to optical shape sensing and more particularly to systems and methods which employ optical shape sensing and sensory feedback for medical applications.
Breathing motion can be a problem in any thoracic or abdominal imaging procedure because, e.g., standard image reconstruction algorithms usually implicitly assume a static scan object. Motion therefore can affect imaging of larger volumes when the time needed for the imaging procedure is comparable to or even longer than the respiratory period. Image-guided interventional procedures can suffer from the effect of motion during the intervention, since this can make automated assignment of reference points in the live images difficult.
Devices for monitoring respiratory motion such as spirometers or breathing belts can yield only crude information about the patient's breathing motion state at specific points in time during the acquisition. With the increasing accuracy of medical imaging modalities and the emergence of new applications such as image-guided interventions, precise knowledge of both the state of the respiratory cycle and shape of body contours and surfaces at a given time becomes increasingly important. Even if breathing is accounted for in images, it may not be sufficient feedback for a physician attempting an operative task, such as inserting a needle or the like.
Making pre-operative information (e.g., patient specific anatomy) effectively available during a procedure is a challenging task in image-guided interventions. Furthermore, having direct, real-time feedback of certain interactions (e.g., needle insertion) during a procedure can assist a physician in decision making or timing of his actions.
In accordance with the present principles, a system for providing sensory feedback includes a garment configured to flexibly and snuggly fit over a portion of a subject. The garment includes one or more sensors disposed therein to monitor activity of the subject or monitor points of interest of the subject. An interpretation module is coupled with the sensors to receive sensor signals and interpret the sensor signals to determine if conditions are met to provide feedback signals to the garment. A feedback modality is incorporated into the garment and is responsive to the feedback signals such that the feedback modality emits energy from the garment to provide sensory information to assist a physician during a procedure. The feedback modality may be responsive to feedback signals from sensors on the garment or sensors looked elsewhere, e.g., on other equipment, from an interventional devices, etc.
A garment providing sensory feedback includes a fabric configured to fit over at least a portion of a subject. One or more sensors are incorporated in the fabric to perform measurements from the subject. A feedback modality is incorporated into the fabric and is responsive to one or more feedback signals derived from measurements such that the feedback modality emits energy from the fabric to provide sensory information to assist a physician during a procedure. The feedback modality may be responsive to feedback signals from sensors on the garment or sensors looked elsewhere, e.g., on other equipment, from an interventional devices, etc.
A method for providing sensory feedback includes providing a garment configured to fit over at least a portion of a subject, one or more sensors incorporated in the garment to perform measurements from the subject, and a feedback modality incorporated into the garment and being responsive to the one or more feedback signals derived from measurements such that the feedback modality emits energy from the garment to provide sensory information to assist a physician during a procedure; generating the one or more feedback signals derived from the measurements; and activating the feedback modality in accordance with the feedback signals to emit energy from the garment to provide sensory information to assist a physician. These and other objects, features and advantages of the present disclosure will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.
This disclosure will present in detail the following description of preferred embodiments with reference to the following figures wherein:
In accordance with the present principles, a measurement system and method provide an adaptable and optimized setup configured to provide status feedback of a patient. Lighting sensors or other feedback mechanisms are associated with a patient's body surface (e.g., by including the sensors in a medical vest or other garment) to provide visual/audio feedback to a physician of a physical state or states of the patient and also to account for augmented reality. Augmented reality refers to the use of pre-operatively collected data employed during a present-time procedure. In one embodiment, the data collected preoperatively may be employed to indicate points of interest directly on a patient during a procedure. In particularly useful embodiments, shape sensing technology is employed to determine shape information of a patient's body. The shape sensing data may be collected using a garment equipped with shape sensing technology.
Shape information can be derived from a variety of systems. These include: optical shape interrogation systems (e.g., fiber optic Bragg sensors, Rayleigh scattering, Brillouin scatter, optical intensity-based attenuation), multi-coil arrays for electromagnetic (EM) localization of points on the apparatus, laser scanning systems for three-dimensional surface estimation and optical/acoustic marker/emitter arrays for camera (time-of-flight or conventional optical measurement) or microphone-based interrogation of shape. Real-time imaging such as ultrasound may also be used for shape information, but the clinical viability of that approach depends on additional cost and clinical value of tomographic information relative to the imaging performed.
A shape sensing garment may be equipped with sensors and/or feedback devices to indicate positions on the patient or a status or activity of the patient (e.g., breathing cycles, swallowing, muscle twitching, etc.). In a particularly useful embodiment, one or more shape sensing optical fibers are employed in a garment with electrical, thermal or other measurement sensors. The fibers and other sensors work in conjunction to provide signals to feedback devices such as light emitting diodes or other feedback mechanisms to give guidance to the physician using preoperative data or currently measured data.
It should be understood that the present invention will be described in terms of medical instruments; however, the teachings of the present invention are much broader and are applicable to any instruments employed in tracking or analyzing complex biological or mechanical systems. In particular, the present principles are applicable to internal tracking procedures of biological systems, procedures in all areas of the body such as the lungs, gastro-intestinal tract, excretory organs, blood vessels, etc. The elements depicted in the FIGS. may be implemented in various combinations of hardware and software and provide functions which may be combined in a single element or multiple elements.
The functions of the various elements shown in the FIGS. can be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions can be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which can be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and can implicitly include, without limitation, digital signal processor (“DSP”) hardware, read-only memory (“ROM”) for storing software, random access memory (“RAM”), non-volatile storage, etc.
Moreover, all statements herein reciting principles, aspects, and embodiments of the invention, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future (i.e., any elements developed that perform the same function, regardless of structure). Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein represent conceptual views of illustrative system components and/or circuitry embodying the principles of the invention. Similarly, it will be appreciated that any flow charts, flow diagrams and the like represent various processes which may be substantially represented in computer readable storage media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
Furthermore, embodiments of the present invention can take the form of a computer program product accessible from a computer-usable or computer-readable storage medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable storage medium can be any apparatus that may include, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read/write (CD-R/W) and DVD.
Referring now to the drawings in which like numerals represent the same or similar elements and initially to
Memory 116 may store an interpretation module 115 configured to interpret electromagnetic, optical, acoustic, etc. feedback signals from a sensitized flexible garment 106. The garment 106 may include fiber sensors, optical, acoustic, electrical or electromagnetic markers or sensors, etc. embedded therein with known geometry or with a geometry that is initialized before use. A shape interrogation console 122 measures the marker/sensor distribution over the surface of interest and supplies feedback about calibration/reference sections and measurement sections to the interpretation module 115. In one embodiment, the shape interrogation module 122 sends and receives light to/from optical fibers or provide electrical power or signals to sensors 104.
When the sensors 104 include optical shape sensing fibers, the optical fiber sensors 104 are weaved or otherwise integrated into garment 106 in a pattern that allows for stretching of the underlying textile substrate while accounting for the fact that the overall fiber sensor length in the textile can change only minimally (e.g., a 2D spiral pattern or 2D sinusoidal pattern embedded within the flexible membrane). The fibers for sensors 104 are locally anchored at control points to provide a strain in the fiber during the flexure of the subject 148. Several control points can constrain the fiber in all degrees of freedom relative to the mesh, e.g., at the fiber tip, whereas others can allow for a sliding degree of freedom so that the fiber can slide freely relative to the mesh pattern to accommodate any overall path length changes in the patterned structure as the mesh deforms.
The interpretation module 115 may include the capability of receiving multiple inputs from multiple devices or systems to interpret an event or dynamic occurrence during a medical procedure, diagnostic test, etc. A medical imaging device 110 and/or a tracking module 117 may also be included and may provide additional feedback to the interpretation module 115. The interpretation module 115 is configured to use the signal feedback (and any other feedback) to account for errors or aberrations related to dynamic changes of a patient's body.
In one embodiment, a subject 148 or a region of interest 140 on the subject 148 is covered or constrained by the flexible garment 106. The flexible garment 106 may include a fabric or netting configured to stretch corresponding with movement or flexure of the subject 148 or the region of interest 140. In addition to sensors 104, garment 106 includes a feedback modality or feedback mechanisms 108. The feedback mechanisms 108 are configurable to react to stimuli collected by sensors 104 (or by other external data). In one example, feedback mechanisms 108 include lights, such as light emitting diodes (LEDs) 109 integrated into the fabric of garment 106. The LEDs 109 are distributed within and throughout the garment 106 to provide feedback as to locations and//or events occurring relative to the patient. Some non-limiting examples of operation of the LEDs 109 (or other feedback mechanisms 108) will now be illustratively described.
In one application, an interventional procedure is performed. For example, a medical device 102 may include, e.g., a needle, a catheter, a guide wire, an endoscope, a probe, a robot, an electrode, a filter device, a balloon device or other medical component, etc. The device 102 is to be inserted in the patient. The device 102 has its coordinate system registered to pre-operative data 119 (e.g., image data). As the device 102 is deployed inside the patient, an LED 109 nearest to a tip of the device 102 (which can be tracked by a tracking device 107) is illuminated on the garment 106 to provide visual feedback to the physician and any other person present in the environment. One or more tracking devices or cameras 107 may be incorporated into the device 102, so tracking information can be provided. The tracking devices 107 may include electromagnetic (EM) trackers, fiber optic tracking, robotic positioning systems, cameras, etc.
In another application, breathing, heatbeat and swallowing data is collected by sensors 104 in the garment 106. The patient's breathing is visually indicated by a first LED (e.g., a white LED illuminated at each breathing cycle), the patient's heartbeat is indicated by a second LED (e.g., a red LED that is illuminated at each beat) and the patient's swallowing is indicated by a third LED (e.g., a blue LED that is illuminated at each swallow).
In yet another application, the garment 106 is spatially registered with pre-operative data 119, such as pre-operative images. In a procedure like a biopsy, image landmarks may be indicated using LEDs 109 to assist in locating a proper insertion point, in addition a depth of the needle may be indicated on the garment 106 by a number or color of LEDs 109 that have been lit.
The system 100 can be configured to be sensitive to any event or movement of a patient. The sensors 104 are configured to relay positional, temperature, electric field information, shape information, etc. back to the interpretation module 115. For example, the garment 106 may be disposed over a mid-section of a patient (148) such that during a breathing cycle sensors 104 sense the dynamic shape changes of the abdomen or chest. This information may be interpreted using the interpretation module 115 which computes distances or changes in distances between nodes or positions in the garment 106. The mesh deflections may then be employed to account for breathing in images taken by an imaging device 110 or assist in the timing of an action during a medical procedure (e.g., inserting a device on an exhale, etc.), or any other event or action that needs compensation for dynamic changes.
It should be understood that the garment 106 may be applied to any portion of the subject's anatomy to collect dynamic data. For example, the garment 106 may be placed over the arms, legs, abdomen, chest, neck, head, or combinations thereof. In addition, the garment 106 may be made adjustable to be configured for different sized subjects, different sized appendages, etc.
The garment 106 may be employed during a medical procedure to assist a clinician in performing the procedure. Workstation 112 may include a display 118 for viewing internal images of the subject 148 using the imaging system 110. The imaging system 110 may include one or more imaging modalities, such as, e.g., ultrasound, photoacoustics, a magnetic resonance imaging (MRI) system, a fluoroscopy system, a computed tomography (CT) system, positron emission tomography (PET), single photon emission computed tomography (SPECT), or other system. Imaging system 110 may be provided to collect real-time intra-operative imaging data. The imaging data may be displayed on display 118. Display 118 may permit a user to interact with the workstation 112 and its components and functions. This is further facilitated by an interface 120 which may include a keyboard, mouse, a joystick or any other peripheral or control to permit user interaction with the workstation 112.
A controller module 126 or other device is provided to condition signals and to control feedback mechanisms 108. In one example, the controller module 126 may generate control signals to control various controllers, sensors, radiation sources/beams, etc. in accordance with programmed conditions for which feedback is desired. The manner of the feedback response can also be programmed. The controller 126 receives data from the interpretation module 115 and issues commands or signals to the feeddback mechanisms 108. The interpretation module 115 dynamically provides information collected and interpreted from the sensors 104 in garment 106 to the controller 126 to render the feedback to the physician in real-time, which may then be employed in administering medication, making decisions, etc.
Referring to
The sensing fibers 210 may include one or more fiber optic Bragg gratings (FBG), which are a segment of an optical fiber that reflects particular wavelengths of light and transmits all others. This is achieved by adding a periodic variation of the refractive index in the fiber core, which generates a wavelength-specific dielectric mirror. A FBG can therefore be used as an inline optical filter to block certain wavelengths, or as a wavelength-specific reflector.
A fundamental principle behind the operation of a FBG is Fresnel reflection at each of the interfaces where the refractive index is changing. For some wavelengths, the reflected light of the various periods is in phase so that constructive interference exists for reflection and consequently, destructive interference for transmission. The Bragg wavelength is sensitive to strain as well as to temperature. This means that FBGs gratings can be used as sensing elements in fiber optical sensors. In a FBG sensor, a shift in the Bragg wavelength, ΔλB is caused, and the relative shift in the Bragg wavelength, ΔλB/λB, due to an applied strain (∈) and a change in temperature (ΔT) is approximately given by:
The coefficient Cs is called the coefficient of strain and its magnitude is usually around 0.8×10−6/μs or in absolute quantities about 1 pm/μs. The coefficient CT describes the temperature sensitivity of the sensor and is made up of the thermal expansion coefficient and the thermo optic effect. Its value is around 7×10−6/K (or as an absolute quantity 13 pm/K). One of the main advantages of the technique is that various sensor elements can be distributed over the length of a fiber. Incorporating three or more cores with various sensors (gauges) along the length of a fiber that is embedded in a structure allows for evaluation of the curvature of the structure as a function of longitudinal position and hence for the three dimensional form of such a structure to be precisely determined.
As an alternative to FBGs, the inherent backscatter in conventional optical fibers can be exploited. One such approach is to use Rayleigh scatter in standard single-mode communications fiber. Rayleigh scatter occurs as a result of random fluctuations of the index of refraction in the fiber core. These random fluctuations can be modeled as a Bragg grating with a random variation of amplitude and phase along the grating length. By using this effect in 3 or more cores running within a single length of multicore fiber, the 3D shape and dynamics of the surface of interest would be trackable.
In one embodiment, the vest 202 may include an integrated lighting arrangement 220 (LEDs) and/or a visual display screen 222 within the vest 202. The lighting arrangement 220 may include a grid or grids of LEDs 224 disposed on a surface or integrated within the vest 202. The grid of LEDs 224 may include or be associated with tracking devices 226 or with the sensing fibers 210 such that each LED 224 can be located relative to the subject 148. The grid pattern shown in
An initialization process may be employed once the vest 202 is securely positioned on the subject 148 to register the location of the vest 202 and/or the LEDs 224 (or display screen 222) with positions or references on the subject 148. If augmented reality is being employed, a registration scheme is needed between pre-operative data (e.g., images) and intra-operative patient positions. Such registration may employ known technologies and methods. For example, fiducial markers on the patient and/or in the vest 202 may be registered with corresponding points in the pre-operative images. The signals from the sensors in the vest 202 are interpreted by the shape interpretation module (module 115), which generates signals that are addressed to one or more locations in the grid of LEDs 224 by the controller 126 (
In order to bridge the gap between the pre-operative information (in particular, the anatomical information provided by a computed tomography (CT) or a magnetic resonance (MR) scan together with associated plans, e.g., needle insertion points on the body surface) and the live view of the physician in the operating room, augmented reality is employed. This may entail a camera system mounted on a head of the physician together with a special set of glasses that have been designed so that while looking through the glasses the physician can virtually overlay preoperative information over a patient or region of interest. This feature can be switched on or switched off. However, the drawback is that the additional camera system and glasses have not been embraced by physicians. In accordance with one embodiment, relevant pre-operative information (images) may be displayed directly on the body surface by making use of vest 202 with integrated lighting or display screen 222, which is worn by the patient throughout the procedure.
This display screen 222 may include a flexible display integrated with the fabric of vest 202. In this embodiment, preoperative images may be registered to the patient and displayed on the screen 222. Representations of instruments or overlays may be generated on the screen in accordance with information received from the sensors to integrate visual feedback via integrated display/lighting components. Depending on the application, the visual feedback feature(s) of the vest 202 may be used independently of the shape sensing.
The LEDs 224 provide visual feedback in the form of information encoded spatially and temporally via color, intensity, phase/timing, direction, etc. Depending on the application these LEDs 224 can be spread sparsely or densely. Information can be presented to reflect guidance information for navigation of an instrument through an access port based on real time measurements of surface anatomy deformation (that in turn reflects organ and target motion relative to an original planned path). Other sensory information may be provided to the physician and can take the form of acoustic or haptic/tactile feedback that is spatially modulated over the surface of the vest/manifold 202.
In one embodiment, knowing the deformation of the outer body surface is of wide interest for many applications. One application includes respiratory motion compensation which is a problem for many image guided interventions. Another application is the deformation of organs and tissue due to applied forces during the interventions (e.g., needle insertion). The vest provides effective feedback, e.g., through visualization. The visual feedback of deformation measurements makes use of lighting from LEDs 224 or from the display screen 222 included in the clothing. The sensor/feedback enabled clothing itself may take the form of a surgical drape or other sterile manifold disposed over the patient. This component may be reusable or disposable in nature, depending on the application and costs involved.
The use for this application would be to convert a measurement from shape sensing fibers into an LED signal. By doing so, the physician would have direct visual feedback, e.g., how much deformation he caused when he inserted the needle or where high areas of respiratory motion exist.
For an augmented reality application, a pre-procedural scan of the patient is acquired. During the intervention, the patient wears the shape sensing vest 202 with the lighting system included. Based on the measurement from the shape sensing, a continuous mapping of the outer surface can be calculated between the pre-operative data and the current patient deformation. This mapping also gives an estimate of the internal organ position (plus locations of derived information, e.g., during planning as a region of interest or insertion point). The lighting system LEDs 224 and/or display screen 222 are employed to display useful information during the intervention or other procedure, e.g., to activate certain LEDs 224 to display the insertion points or even to use the LEDs 224 to display the organ position mapped on a surface of vest 202. Different colors of the LEDs 224 could also improve the visual feedback. The shape sensing vest 202 permits tracking of patient/organ deformation in real-time. The LED feedback is able to account for, e.g., respiratory motion, reactionary forces, etc. Display screen 222 may include a higher-resolution flexible display integrated in the vest 202 for displaying medical (imaging) data. The medical data can be filtered or transformed depending on input of the deformation sensors 210 in the vest 202 or other data acquired by other sensors, e.g., location of a needle tip. The display screen 222 and/or LEDs 224 may indicate medical alarms and/or sites where complications may have occurred. The vest 202 may be employed to display medical condition diagnoses depending on other sensors internal or external to the vest 202.
Referring to
Instead of or in addition to a vest, another form of clothing may be employed for garment 302, e.g., a pair of trunks, a bra, a skull cap, a sock, a glove, etc. The clothing may include elastic, Spandex™ or other form of elastic clothing. The garment 302 (or any other garment in accordance with the present principles) may include adjustment mechanisms 304 to be adaptively sized to snuggly fit a patient. The adjustment mechanisms 304 may include hook and loop connectors, buckles, elastic bands, zippers, snaps, adhesive strips, inflatable cuffs, suction devices or any other devices for connecting to a body surface and/or adjusting to a size.
Referring to
In block 404, one or more feedback signals are generated. The feedback signals are preferably derived from measurements of the one or more sensors. Other sensors or devices may also be employed to trigger the generation of feedback signals. These sensors or devices may include monitoring devices or stored data not necessarily located on the garment. In block 406, the measurements of the one or more sensors are interpreted to determine one or more of anatomical movement, anatomical function, a position of an interventional device, etc.
In block 408, the feedback modality is activated to emit energy from the garment to provide sensory information to assist a physician. In block 410, lights or other devices may be selectively illuminating in an array to indicate, e.g., a location of an anatomical feature, anatomical movement, pre-operative data, a position of an interventional device, etc. In block 412, a display screen may be illuminated to render images to indicate a location of an anatomical feature, anatomical movement, pre-operative data, a position of an interventional device, etc. The images may include pre-operative images.
In block 414, decisions are made, procedures conducted, etc. using the sensory feedback generated by the garment. The sensory feedback may be employed by physicians to determine a patient status, determine triggering events, understand the boundaries of an organ, obtain anatomical responses, etc.
In interpreting the appended claims, it should be understood that:
Having described preferred embodiments for systems and methods for body surface feedback for medical interventions (which are intended to be illustrative and not limiting), it is noted that modifications and variations can be made by persons skilled in the art in light of the above teachings. It is therefore to be understood that changes may be made in the particular embodiments of the disclosure disclosed which are within the scope of the embodiments disclosed herein as outlined by the appended claims. Having thus described the details and particularity required by the patent laws, what is claimed and desired protected by Letters Patent is set forth in the appended claims.
Related application is “Motion Compensation and Patient Feedback in Medical Imaging Systems,” PCT IB2011/051340, Mar. 29, 2011.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/IB2012/055729 | 10/19/2012 | WO | 00 | 4/18/2014 |
| Number | Date | Country | |
|---|---|---|---|
| 61549428 | Oct 2011 | US |
