Patent Application
20200022657
The present disclosure generally relates to systems and methods for analyzing data gathered during a minimally invasive procedure, such as an invasive cardiology or electrophysiology procedure, and more specifically relates to an analysis and review system providing time synchronized display of multiple datasets, including, but not limited to, data collected by an electrophysiology system and image data captured by one or more imaging devices. Minimally invasive procedures, such as those performed in a cardiac catheterization laboratory, involve the collection of data relating to multiple different modalities, including measurement data from sensors in catheters, from physiological recording devices or modalities (e.g., ECG surface electrodes, physiological electrodes in catheters, invasive and noninvasive blood pressure monitors, respiration monitors, electroencephalographs, SpO2 monitors, etc.) and from imaging devices (e.g., ultrasound, x-ray, computed tomography, magnetic resonance, nuclear (PET), 3D mapping, or optical CT imaging devices). Many different minimally invasive procedures may be performed, such as in the catheterization laboratory, utilizing some or all of the foregoing devices and systems, including angiography studies, electrophysiology studies, stent placements, and cardiac ablation, to name a few.
The x-ray images are acquired using cardiovascular x-ray imaging equipment. The resulting images are stored in digital form as DICOM (Digital Imaging and Communications in Medicine) images and stored and viewed electronically. These digital images are available for review and analysis at a physician review workstation.
During minimally invasive procedures, the patient also undergoes physiological recording modalities using a hemodynamic recording system. The hemodynamic recording system hooks up to a patient via externally placed leads that monitor the electrical impulses from the heart and records the heart's electrical activity in the form of a waveform. This record, called an electrocardiogram (ECG), is analyzed by well-known software that measures the heart's rhythms and electrical impulses, allowing the physician to detect heart irregularities, disease and damage. The ECG data, including waveforms and results of analysis, is typically stored in a computer database.
Additional data sources include electrodes on catheters inserted into the patient to measure electrical activity from within the heart, as well as a number of other types of physical modality sensors on catheters, including pressure sensor, temperature sensors, and current sensors.
This Summary is provided to introduce a selection of concepts that are further described below in the Detailed Description. This Summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used as an aid in limiting the scope of the claimed subject matter.
In one embodiment, a minimally invasive procedure analysis and review system includes a display device, a user input device, a computing system communicatively connected to the display device and the user input device, and a study analysis module executable on a processor. The study analysis module is configured to receive a running tally of events during the minimally invasive procedure, wherein each event includes an event time and an event type. An event is selected from the running tally of events, and at least two relevant datasets are determined based on the event type of the selected event. A relevant time portion of each relevant dataset is identified based on the event time of the selected event, and the relevant time portions of each of the at least two relevant datasets is displayed on the display device.
One embodiment of a method of operating a computing system to facilitate analysis and review of datasets collected during a minimally invasive procedure on a patient, wherein a computing system provides a graphical user interface on a display device and receives input from a user input device, includes receiving a running tally of events during a minimally invasive procedure, wherein each event includes an event time and an event type. A selected event is received from the running tally of events, and at least two relevant datasets are determined based on the event type of the selected even. A relevant time portion of each relevant dataset is then identified based on the event time of the selected event, and the relevant time portions of each of the at least two relevant datasets is displayed on a display device.
Another embodiment of a method of operating a computing system to facilitate analysis and review of datasets collected during a minimally invasive procedure includes providing multiple datasets, one for each of multiple modalities collected during the minimally invasive procedure, wherein all of the datasets are time synchronized to a reference clock, and then identifying at least two relevant datasets for display out of the multiple datasets. A selected time period is identified according to the reference clock, and a relevant time portion of each relevant dataset is identified based on the selected time period. The relevant time portions of each of the at least two relevant datasets is then displayed on a display device. A user input is received to adjust the selected time period, and an adjusted selected time period is identified based on the user input and according to the reference clock. An updated relevant time portion of each relevant dataset is identified to include data occurring during the adjusted selected time period, and the display for each of the at least two relevant datasets is adjusted to display the updated relevant time portions of each of the at least two relevant datasets on the display device.
Various other features, objects, and advantages of the invention will be made apparent from the following description taken together with the drawings.
The present disclosure is described with reference to the following Figures.
The present inventors have recognized that current minimally invasive procedures involve a multitude of devices each providing data that needs to be accounted for during various portions of the procedure. Accordingly, the inventors have recognized that clinicians performing or involved in minimally invasive procedures suffer from information overload and fatigue, especially in electrophysiology studies. Accordingly the inventors have recognized that systems and methods are needed for associating disparate data sources, assessing differing technical standards, and monitoring the operation of multiple data collection systems. Moreover, the inventors have recognized that an analysis and review system is needed that correlates and presents relevant sources of data in a time-correlated manner and in association with thorough and domain-aware event marking and analysis.
The inventors have further recognized that currently available systems present the various datasets independently. For example, images collected during the minimally invasive procedure are presented separately from physiological signal data, which are presented separately still from catheter data, such as from temperature sensors, pressure sensors, or current sensors on a catheter. Thus, using current systems to observe, review, and make determinations based on the multitude of available data is a laborious process and can be time intensive. Often, clinicians have limited time during procedures to assess datasets and make decisions. Accordingly, the inventors have further recognized that, given the data overload and difficulty of navigating the various data sets, clinicians are too often unable or too overloaded to sufficiently review the data in order to make an informed decision during a procedure.
In view of the forgoing problems and challenges with current systems recognized by the inventors, they developed the disclosed analysis and review system for analyzing multiple different types of datasets from different modalities collected during a minimally invasive procedure—e.g., from any of various sensors on catheters, from various physiological recording devices used to gather physiological information about the patient, from the patient's medical history set forth in their medical record, and/or data entered in by a clinician during a procedure (such as lab results or clinician observations about the patient). Each dataset includes data values—e.g., physiological measurements, image data, sensed pressure or temperature values, lab test values—and parameter values. Parameter values describe information about the dataset, such identifying the device that collected the data, physical conditions or settings of the collection device (e.g., mode settings, position of the c-arm or other x-ray device, x-ray intensity). Parameter values also include a time parameter indicating relevant time, such as a time value for each data value in a physiological dataset.
The procedural analysis and review system selects one or more relevant datasets in a time-synchronized and correlated way. Relevant datasets are datasets from the multiple available datasets that are identified as relevant based on the problem domain and/or based on clinician input. For example, relevant datasets may be identified based on an event selection by a clinician from the running tally of events in the procedure. The relevancy determination is made based on one or more of the parameter values for that dataset, such as by identifying the parameter values that are correlated to the event type of the selected event.
The relevant datasets may be identified based on a user-selected dataset or modality, such as selecting and controlling a time period of a particular dataset of the user-selected dataset. In other embodiments, the relevant datasets may be identified based on a user-selected event. For example, the relevant datasets may be identified based on a user-selected event from a running tally of events identified during the procedure. In either embodiment of relevant dataset identification, the system adjusts and updates review windows of all relevant datasets together, such that an adjustment to the time period in one of the relevant dataset windows is reflected in the displays of all of the other relevant datasets as well so that all of the modality windows correlate with the selected time period.
For example, the disclosed analysis and review system may analyze and intelligently present data collected by an electrophysiology system and by one or more imaging devices, such as x-ray imaging devices and/or ultrasound imaging devices, during the minimally invasive procedure. The analysis and review system receives and analyzes time-sequenced information and correlates the data in time. The system further catalogs and recognizes events, such as clinician marked events, programed, or procedure, events, or events detected based on the data itself (such as when one or more of the data values for one or more datasets exceeds a relevant threshold). Thereby, the disclosed system and method provides a means for efficiently observing and reviewing multiple data sets over a procedure period and for identifying and reviewing the most important and relevant data in order to make informed determinations during the procedure. Similarly, the disclosed system and method provide clinicians the ability to review time-synchronized images and data, and to navigate between physiological data and information collected throughout the procedure, to provide a comprehensive review of all the datasets within one uniform and easy-to-navigate user interface environment.
Each of the multiple datasets provided to the computing system 200 are time-stamped in a way that the various datasets can be time-correlated. In one embodiment, all of the datasets is time synchronized to a reference clock, and the running tally of events identified during the procedure is also organized according to the same reference clock so that all data and events can be correlated accordingly by the system 1. In various embodiments, the reference clock 28 may be provided in the computing system 200, as in the depicted embodiment, or may be provided and/or associated with the DICOM server 24. In one embodiment, each frame of image data acquired by an imaging device 20, 22 is time-stamped with local time according to a clock located in or associated with the respective imaging device 20, 22. The time stamp may be embedded or encapsulated in the image data file, or object, such as inserted into a predetermined field in a header in the DICOM object. Similarly, the data collected during the procedure by the electrophysiology system 10 is also time-stamped according to a local clock in the electrophysiology system 10, such as a clock associated with the ADC 16. Each local clock located in each respective imaging device 20, 22 and the electrophysiology and vascular recording system 10 is then synchronized to a reference clock 28, which in various embodiments may be provided in the DICOM server 24 or in the computing system 200. For example, each imaging device 20, 22 and the electrophysiology and vascular recording system 10 may be configured to determine respective offsets of its local clock relative to the reference clock 28. Alternatively, the computing system 200 may be configured to determine and monitor the respective offsets for the local clocks and each of the associated systems and devices 10, 20, 22. The offsets may be stored in association with each data file or object, such as in a designated offset field in the header in the DICOM object or other file-type object. An exemplary system for synchronization of cardiovascular images and physiological or hemodynamic recording data is described at U.S. Pat. No. 7,280,864, which is hereby incorporated by reference in its entirety.
In still other embodiments, the various datasets may be correlated by other means, such as according to one of the local clocks. Thus, for example, offsets may be determined between the clock associated with the electrophysiology and vascular recording system 10 and each of the local clocks associated with the imaging devices 20, 22. For example, each of the local clocks in the electrophysiology and vascular recording system 10 and the imaging devices 20, 22 may be adjusted in real-time using network time protocol (NTP) time synchronization or by some other time synchronization protocol, which may be to a separate designated reference clock 28, or to a respective one of the local clocks.
The computing system 200 comprises a study analysis module 6, which is software module executable to identify one or more datasets to be displayed and to display thos datasets in a time-correlated manner. In certain embodiments, the computing system 200 may also comprise one or more event recognition modules 8 configured to assess the datasets, such as from the catheters 12 and/or surface leads 14, to detect events. For example, the event recognition module 8 may be executable to assess physiological signal data and catheter data from the electrophysiology and vascular recording system 10 and identify a threshold triggered event when the physiological signal data for one or more physiological modalities exceeds a relevant physiological threshold, or when the catheter data for one or more catheter measurement modalities exceeds a relevant physical measurement (e.g., pressure, temperature, etc.) threshold set for a respective catheter modality. One example in EP might be to monitor the invasive blood pressure channel, which is often used with a transseptal needle, to indicate when the septum has been breached. This is done to enable a sheath to be inserted to allow catheters to pass to the left side of the heart. The event recognition module 8 may be configured to detect and highlight the time of a threshold pressure change, or reaching a threshold pressure, and to highlight when it was possible to cross between chambers. The recognized event is assigned an event type based on, for example, the pressure sensing modality and/or the time in the procedure where the threshold pressure change occurred. The pressure event type is associated with certain parameter values within the system—e.g., ultrasound, x-ray, iECG, etc. Thereby, the system 1 utilizes domain knowledge to assist in navigating through the procedure to locate and automatically display data associated with a particular selected event.
Each threshold triggered event detected by the event recognition module 8 may further include an event time and an event type. For example, the event recognition module 8 may be configured to assign the event time based on a time that the relevant threshold was exceeded, such as according to the reference clock 28. The event recognition module 8 may further be configured to determine the event type based on the modality and relevant threshold that was exceeded. For example, the event recognition module 8 may be configured to recognize an event when one or more temperature measurements from a temperature sensor on a catheter exceed a temperature threshold, when one or more pressure measurements from a pressure sensor on a catheter exceed a pressure threshold, and/or when one or more current measurements from a current sensor on a catheter exceed a current threshold. For instance, esophageal temperature monitoring data from an esophageal temperature probe may be analyzed during ablation, such as to assess whether the temperature data exceeds a relevant threshold or threshold change, and to detect an event accordingly. Marking such an event is important because it is possible to create a serious complication in ablation where excessive heating can cause lesions in the esophagus. Relevant parameters for the esophageal temperature event might include, for example, Thereby, temperature events related to esophageal temperature monitoring could be searched and selected, and the displayed datasets will automatically populate accordingly.
Alternatively or additionally, the event recognition module 8 may be configured to recognize and/or store other types of events. For example, the event recognition module 8 may be configured to recognize events, including determining an event time and an event type, based on user inputs to the system, such as user inputs via the user interface 30 and/or inputs to control one or more catheters 12 in the electrophysiology and vascular recording system 10. For example, the event recognition module 8 may be configured to recognize and document one or more procedure events marking occurrence of a step in the minimally invasive procedure, such as any patient preparation step, medication delivery step, catheter insertion step, ablation or stent placement, or the like. For example, a procedure event may be triggered based on a user input, such as a macro-input where a single input is associated with and initiates execution of multiple instructions. One of the instructions associated with the macro may be recognition of a particular procedure event by the event recognition module 8.
Similarly, the event recognition module may further be configured to identify one or more clinician marked events based on user input by a clinician via the one or more user input devices 32 to mark a particular time. Depending on the clinician input, the clinician marked event may mark a particular time in a particular modality data stream, or may mark a particular time in the overall procedure. Accordingly, the event recognition module 8 will determine an event type based on the form of the user input, such as whether the user input is provided to mark time in a single dataset or to mark a time in the procedure. Clinician marked events may be based on user input provided by a clinician to mark a dataset or an event in the procedure real-time as the event is occurring or the data is being collected, or the clinician input to generate a clinician marked event may be provided by the clinician while the clinician is reviewing previously-recorded datasets. For example, a clinician marked event may be a “bookmark” providing a time marker that can be utilized by a clinician during review to locate a particular point in the data. Such review commonly occurs during minimally invasive procedures, such as to make decisions on whether further studies or imaging are needed, or whether an intervention is warranted. Additionally, the user input to generate a clinician marked event may occur during post-procedure review, such as when a clinician is reviewing a procedure for documentation purposes.
The study analysis module 6 and event recognition module 8 receive datasets, including physiological signal data, catheter data, image data, etc., and generate a time-synchronized and domain-aware user interface through which a clinician can review all datasets collected during a minimally invasive procedure, and can facilitate data analysis by identifying relevant datasets based on a number of factors, including events that have occurred in a procedure, datasets collected, a type of procedure, a current point in a procedure, user input by a clinician, etc.
In the example at
The computing system 200 is also configured to receive various user inputs. In the depicted example, the computing system receives macro user inputs 50, which as described above, are single instructions that expand automatically into a set of instructions to perform a set of tasks. Event marker user inputs 52 may also be provided, such as a clinician providing input via the user input devices 32 marking and event within a particular dataset or within the procedure as a whole, as is described above. Event selection user input 54 is also provided, which is user input to select one or more events from the running tally of events 70. Dataset selection user input 56 may also be received, which is user input to select a dataset or modality. The dataset selection user input 56 may be for selecting a dataset to be viewed within the analysis and review system 1, or to select a dataset for inclusion in the relevant datasets 71. A user may also provide time period selection user input 58 to select or adjust the time period of data displayed by the system 1. Additionally, the user may provide a point-in-time selection user input 60 to mark or select data at a particular point-in-time. For example, a point-in-time selection user input 60 may be utilized to select a particular point-in-time within one dataset that will be reflected in the display of the other relevant dataset data sets. To provide just one illustrative example, the user may provide a point-in-time selection user input 60 to select a point within the catheter data 42, which may cause display of corresponding image data captured at the selected point-in-time. Conversely, the user may provide a point-in-time selection user input 60 to select a point-in-time in the image data 44, 46, 48, which will cause the display to identify corresponding physiological signal data 40 and/or catheter data 42 to be visually identified on the display, depending on the current relevant datasets 71.
As described above, the event recognition module 8 recognizes events based on user input (e.g., event marker user input 52) and/or based on the data itself, and each recognized event gets added to the running tally of events 70, which is one output of the system. The study analysis module 6 generates and facilitates displaying of the data via the user interface 30. For example, the study analysis module 6 identifies relevant datasets 71 for display on one or more display devices 34 comprising part of the user interface 30. The study analysis module 6 further identifies a relevant time portion of each relevant dataset, and displays the relevant time portions of each relevant dataset 71 in a time-coordinated fashion so that the relevant time periods of the displayed data correspond and represent the same time period across all displayed datasets. As a user navigates through one displayed dataset, all other displays of data will update accordingly. Thus, if a user changes the period of time displayed for one dataset, the displays of all other datasets will be updated to display that same period of time. Thus, all datasets are displayed in a time-correlated, or time-locked, fashion so that the same time period is displayed across all of the review panes showing the various relevant datasets.
The study analysis module 6 may be configured to identify the relevant datasets 71 based on user input, such as datasets selected by a user. For example, the study analysis module 6 may identify relevant datasets 71 based on the datasets that a user is currently viewing. Alternatively or additionally, the study analysis module 6 may prompt or allow a user to provide input to select relevant datasets. In still other embodiments, the study analysis module 6 may automatically determine one or more relevant datasets 71 to be displayed, which may not be directly selected by a user, but are still based on certain user input. For example, a user may select an event or set of events within the running tally of events 70 (e.g., event selection user input 54), and the study analysis module 6 may determine the relevant datasets 71 based on the selected event, such as based on the event type of the selected event. For example, the study analysis module 6 may have a look-up table or other association table associating each of the various possible event types with a set of relevant parameter values, which are then used to identify the relevant datasets 71. To provide just one example, the event type may be associated with parameter values indicating a list of relevant data recording modalities—e.g., pressure sensor, ultrasound, and iECG—and the relevant datasets whose parameter values match those recording modalities are then identified.
Alternatively or additionally, the study analysis module 6 may select a set of relevant datasets 71 based on a selected portion of a particular dataset. For example, if a clinician views a portion of a particular dataset, the study analysis module 6 may be configured to select a set of relevant datasets 71 for display in conjunction with the selected portion of the dataset being viewed (e.g., by identifying key events occurring in the selected time period that relate to the dataset being viewed). For example, such an action by the study analysis module 6 may be triggered upon user input to engage the time-correlated analysis and review mode.
Once the relevant datasets are identified, the study analysis module 6 automatically displays those relevant datasets and provides a relevant time portion of each displayed relevant dataset 71.
The relevant time portions 72, 76 may be determined based on the selected event or set of events, such as by identifying an event window that encapsulates a selected event or a selected point-in-time inputted by a user (e.g. via user inputs 54, 56, 58, or 60). The event window may be, for example, determined as a predetermined period of time on either side of a selected event time or a selected point-in-time. Alternatively, the event window may be based on a start time and end time of a selected event or set of events, as illustrated and described below regarding the example at
The selected events 95 include a starting event 95a and an ending event 95b at each of the start time and end time of an ablation portion of a procedure. Between the ablation start 95a and the ablation end 95b, three intervening events are marked. For example, the intervening events may be clinician marked events, procedure events, or threshold triggered events. The review panes 81b-81e provide physiological data, catheter data, and image data occurring during the selected set of events 95 for the relevant datasets. In the depicted embodiment, the relevant datasets are identified based on the selected events 95 (i.e., the ablation events), and are not identified based on the intervening events (which could be related or unrelated to the ablation events). In certain embodiments, the study analysis module 6 may be configured to identify whether any of the intervening events are related to the selected events 95, and if any of the intervening events are related then to further identify the relevant datasets based on the relevant intervening events as well.
In the example at
In the depicted example, a set of representative images 92 between the ablation start and end times are highlighted to designate the relevant time portion of image data. Additionally, an identified image 93 is presented. The identified image 93 corresponds with the adjustable time marker 85 presented across all of the review panes identifying a particular point-in-time within the relevant time portion of a dataset. The identified image 93 is an image occurring at the selected point-in-time. In certain embodiments, each of the adjustable time markers 85a-85e can be moved by a user in order to adjust the time portions of the dataset displayed. Moving any one of the adjustable time markers 85a-85e will cause the markers in the remaining review panes to also be adjusted accordingly Likewise, the adjustable time marker 85d in the image review pane 81d may be adjusted by playing the identified image 93, and the adjustable time markers 85b and 85c will move correspondingly to designate the point-in-time in the respective dataset that correlates with the image, or video frame, being played in the identified image window 93.
Once a threshold triggered event is detected at step 134, the event time and event type are recorded. The event time is determined at step 136 based on the time of the data value that exceeded the relevant threshold. For example, the event time may be identified as the time of the data value according to the reference clock. In certain embodiments, that may be local time stamp associated with the data value plus any offset for correlating the local clock to the reference clock. The event type is then determined at step 138 based on the physiological modality or catheter modality that exceeded the threshold. Alternatively or additionally, the event type may also be determined based on the relevant threshold that was exceeded, such as whether a low threshold or a high threshold for the relevant dataset was exceeded. Depending on the type of data (e.g., the modality represented by the data), exceeding a low threshold may be assigned to a different event type than exceeding a high threshold. The threshold triggered event is than added to the running tally of events at step 140.
Referring again to
Although the computing system 200 as depicted in
The processing system 206 includes the processor 207, which may be a microprocessor, a general purpose central processing unit, an application-specific processor, a microcontroller, or any other type of logic-based device. The processing system 206 may also include circuitry that retrieves and executes software 202 from storage system 204. Processing system 206 can be implemented within a single processing device but can also be distributed across multiple processing devices or sub-systems that cooperate in executing program instructions.
The storage system 204 can comprise any storage media, or group of storage media, readable by processing system 206, and capable of storing software 202. The storage system 204 can include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Storage system 204 can be implemented as a single storage device but may also be implemented across multiple storage devices or sub-systems. Examples of storage media include random access memory, read only memory, optical discs, flash memory, virtual memory, and non-virtual memory, magnetic sets, magnetic tape, magnetic disc storage or other magnetic storage devices, or any other medium which can be used to store the desired information and that may be accessed by an instruction execution system, as well as any combination or variation thereof, or any other type of storage medium. Likewise, the storage media may be housed locally with the processing system 206, or may be distributed in one or more servers, which may be at multiple locations and networked, such as in cloud computing applications and systems. In some implementations, the storage media can be a non-transitory storage media. In some implementations, at least a portion of the storage media may be transitory.
The communication interface 208 interfaces between the elements within the computing system 200 and external devices, such as the user input device 32 and the display device 34 of the user interface 30. Additionally, the communication interface 208 may interface with the DICOM server 24 and/or the electrophysiology and vascular recording system 10 or imaging devices 20, 22.
The user interface 30 is configured to receive input from a user, such as a clinician, via one or more user input devices 32 and to facilitate provision of the graphical user interface 36. User input devices 32 may include a mouse, a keyboard, a voice input device, a touch input device for receiving a gesture from a user, a motion input device for detecting non-touch gestures and other motions by a user, and other comparable input devices and associated processing elements capable of receiving input from a user, such as a clinician. The user interface 30 further includes a display device 34, such as a video display or graphical display that can display a graphical interface as disclosed herein. Speakers, printers, haptic devices and other types of output devices may also be included in the user interface 210.
This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to make and use the invention. Certain terms have been used for brevity, clarity and understanding. No unnecessary limitations are to be inferred therefrom beyond the requirement of the prior art because such terms are used for descriptive purposes only and are intended to be broadly construed. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have features or structural elements that do not differ from the literal language of the claims, or if they include equivalent features or structural elements with insubstantial differences from the literal languages of the claims.
