Patent Application
20190259491
Embodiments described herein relate to systems and methods for populating a PACS database. More specifically, embodiments described herein relate to populating a PACS database based on instance level metadata retrieved from at least one image archive.
A physician diagnosing a patient may order an image study using a particular imaging modality. Imaging modalities use various imaging techniques and hardware, including, for example, radiography, nuclear medicine, computed tomography (CT), mammography, molecular imaging, photoacoustic imaging, magnetic resonance imaging (MRI), echocardiography, magnetic particle imaging, elastography, tactile imaging, and ultrasound imaging.
Medical images collected as part of an imaging procedure and corresponding metadata are stored in an archive as a study. The image archive may be included in or communicate with a picture archiving and communication system (PACS), which controls access to images generated by one or more imaging modalities. A PACS may include a PACS server and a PACS database. The PACS database may store information regarding images stored in a remote archive, such as pointers, that may be used to retrieve an image from a particular remote archive. The PACS server may use the data stored in the PACS database to respond to requests for images from a viewer device submitted through an image viewer application (“image viewer”).
For example, a medical practitioner (for example, a reading physician) may use an image viewer to communicate with a PACS server and request an image study. The PACS server may respond to the request by providing location information for the requested image study, which the image viewer may use to directly access the images from the image archive. The medical practitioner may use the image viewer to review the retrieved images included in the requested study and, optionally, to generate a report containing findings for the image study.
When a medical practitioner retrieves and reviews medical images generated as part of a particular image study for a particular patient, the medical practitioner often also retrieves and reviews images generated as part of other image studies for the patient. For example, a reading physician may retrieve images generated as part of a previous image study to verify findings for a current image study or to better understand the images in the current image study. The previous image study may include images of the same or different anatomy as the current image study, images generated by the same or a different imaging modality as the current image study, or a combination thereof. For example, patients may have had multiple imaging studies performed at different times, which may or may not be related.
For example,
Various relevancy rules may be applied to determine the available image studies or series to list for a medical practitioner with respect a currently-viewed image study. These relevancy rules may consider the imaging modality used for the currently-viewed image study, the anatomy represented in the currently-viewed image study, a finding entered for the currently-viewed image study, or the like. Accordingly, to acquire sufficient data regarding other image studies available for a particular patient to apply the relevancy rules, other image studies may be prefetched, which involves downloading the entire contents (including image data) of a limited range of prior studies (for example, to the PACS database, the image viewer, or both). For example, when a medical practitioner is reading an image study for a particular patient, other images studies for the patient may be prefetched. Given the number of image studies for a particular patient, however, it may be difficult to identify what image studies to prefetch. Also, given the size of some image studies, only a limited number of studies may be prefetched. For example, typically only three to six image studies may be prefetched. Furthermore, prefetching may waste computing resources as an entire study may be downloaded that a reading physician never accesses. For example, conventional prefetching approaches rely on downloading entire studies and then parsing the studies for metadata, which may be used to generate the thumbnails or listings of studies as described above. Thus, this approach is costly in terms of bandwidth and computing requirements.
Alternatively or in addition, a PACS server may populate a PACS database with study level metadata (sometimes referred to providing a “half folder” mechanism). In particular, the PACS server retrieves patient demographics and study level metadata from the remote image archive and populates the PACS database based on the retrieved data. Accordingly, the PACS server may apply the relevancy rules as described above to this stored metadata to determine other image studies that may be relevant. However, even with this information, a medical practitioner typically needs to wait until an entire study is downloaded and processed (for example, parsing and processing of metadata for the images, series, or both included in the image study) before information regarding the image study can be provided to a medical practitioner (for example, in thumbnail form as described above). For example, an image viewer may apply various rules to image metadata to determine how to properly hang a study or an image and how a user can navigate through the images. Accordingly, populating a new PACS database using a “half folder” mechanism still results in delay and wasting of computing and bandwidth resources.
Thus, embodiments described herein populate a PACS database with instance level metadata (sometimes referred to herein as providing a “full folder” population), which can be used to display a useful depth of information to a medical practitioner regarding other image studies for a patient without requiring a full download of individual image studies or series and the computing and bandwidth resources associated with such a download. As described in more detail below, this mechanism retrieves (in addition to patient demographics and study level metadata as described above for the “half folder” mechanism) instance level information (metadata) for image studies of a particular patient from an existing image archive and populates the PACS database with the retrieved information. Accordingly, the PACS database, once populated, maintains the locations (references or pointers) to the instances (images, presentation states, key objects, reports, scanned documents, measurements, etc.), which provide an efficient image retrieval mechanism without the need to query multiple devices. Having the full set of instance level metadata also provides the information needed for efficient “hanging” or display of studies so that the images are arranged (and optionally downloaded) in an efficient and pleasing manner for the reading physician. Thus, by cataloguing not merely the study level metadata but also the instance level metadata, rules can be executed against the retrieved metadata to determine how to properly hang the study, to determine the order of retrieval of the images to the workstation for background loading, to allow a user to navigate the study at will (or at random), and the like without having to wait for the entire study (including one or hundreds of images) to be downloaded and metadata to be parsed after the download is complete. Furthermore, as compared to conventional prefetching as described above, the “full folder” population method described in the present application catalogues metadata for the entire set of studies for a patient (which may be dozens or hundreds of studies), which allows a user to organize and navigate through image studies quickly and efficiently. In particular, the metadata can be used to generate the thumbnails and listing of available image studies as described above, which allows a user to view sufficient detail regarding other image studies (including potentially all other image studies for a particular patient), without requiring a full download of any of the individual image studies.
For example, one embodiment provides a system for populating a target database with metadata for use in accessing medical image data. The system includes an electronic processor communicatively coupled to a memory. The memory stores instructions that, when executed by the electronic processor, cause the electronic processor to, in response to a trigger event, query an archive database via a communication interface for instance level metadata of a plurality of image studies based on at least one query rule and store the instance level metadata of the plurality of image studies received from the archive database in the target database. The memory also stores instructions that, when executed by the electronic processor, cause the electronic processor to, based on an image study for a patient requested through an image viewer and at least one relevancy rule, determine a set of other image studies for the patient based on the instance level metadata stored in the target database for the plurality of image studies and display information regarding the set of the other image studies for the patient within the image viewer based on the instance level metadata stored in the target database without downloading image data for each image included in the set of the other image studies to the target database.
Another embodiment provides a method of populating a target database with metadata for use in accessing medical image data. The method includes, in response to a trigger event, querying, with an electronic processor, an archive database via a communication interface for instance level metadata of a plurality of image studies based on at least one query rule and storing, with the electronic processor, the instance level metadata of the plurality of image studies received from the archive database in the target database. The method also includes, based on an image study for a patient requested through an image viewer and at least one relevancy rule, determining a set of other image studies for the patient based on the instance level metadata stored in the target database for the plurality of image studies and displaying information regarding the set of other image studies for the patient within the image viewer based on the instance level metadata stored in the target database without downloading image data for each image included in the set of other image studies to the target database.
Yet another embodiment provides non-transitory computer-readable medium storing instructions that, when executed by an electronic processor, perform a set of functions. The set of functions including, in response to a trigger event, querying an archive database via a communication interface for instance level metadata and series level metadata of a plurality of image studies based on at least one query rule and storing the instance level metadata and the series level metadata of the plurality of image studies received from the archive database in a target database. The set of functions also includes, based on an image study for a patient requested through an image viewer and at least one relevancy rule, determining a set of other image studies for the patient based on the instance level metadata and the series level metadata stored in the target database for the plurality of image studies and displaying information regarding the set of other image studies for the patient within the image viewer based on the instance level metadata and the series level metadata stored in the target database without downloading image data for each image included in the set of other image studies to the target database.
Other aspects of the invention will become apparent by consideration of the detailed description and accompanying drawings.
One or more embodiments are described and illustrated in the following description and accompanying drawings. These embodiments are not limited to the specific details provided herein and may be modified in various ways. Furthermore, other embodiments may exist that are not described herein. Also, the functionality described herein as being performed by one component may be performed by multiple components in a distributed manner. Likewise, functionality performed by multiple components may be consolidated and performed by a single component. Similarly, a component described as performing particular functionality may also perform additional functionality not described herein. For example, a device or structure that is “configured” in a certain way is configured in at least that way, but may also be configured in ways that are not listed. Furthermore, some embodiments described herein may include one or more electronic processors configured to perform the described functionality by executing instructions stored in a non-transitory, computer-readable medium. Similarly, embodiments described herein may be implemented as non-transitory, computer-readable medium storing instructions executable by one or more electronic processors to perform the described functionality. As used in the present application, “non-transitory computer-readable medium” comprises all computer-readable media but does not consist of a transitory, propagating signal. Accordingly, a non-transitory computer-readable medium may include, for example, a hard disk, a CD-ROM, an optical storage device, a magnetic storage device, a ROM (Read Only Memory), a RAM (Random Access Memory), register memory, a processor cache, or any combination thereof.
In addition, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. For example, the use of “including,” “containing,” “comprising,” “having,” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. The terms “connected” and “coupled” are used broadly and encompass both direct and indirect connecting and coupling. Further, “connected” and “coupled” are not restricted to physical or mechanical connections or couplings and can include electrical connections or couplings, whether direct or indirect. In addition, electronic communications and notifications may be performed using wired connections, wireless connections, or a combination thereof and may be transmitted directly or through one or more intermediary devices over various types of networks, communication channels, and connections. Moreover, relational terms such as first and second, top and bottom, and the like may be used herein solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
As noted above, embodiments described herein provide an instance level or “full folder” population method for a PACS database. The method may retrieve instance level information (metadata) for image studies of a particular patient from an existing archive based on one or more triggering events. The stored instance level metadata provides information needed for efficient “hanging” or display of studies so that the images are arranged (and optionally downloaded) in an efficient and pleasing manner for a medical practitioner. In particular, by cataloguing not merely the study level metadata but also the instance level metadata, rules can be executed against the retrieved metadata to determine how to properly hang the study, to determine the order of retrieval of the images to the workstation for background loading, to allow a user to navigate the study at will (or randomly), and the like without having to wait for the entire study (including one or hundreds of images) to be downloaded and the metadata to be parsed after the download is complete. Thus, embodiments described herein provide novel mechanism for populating a PACS database to provide medical practitioners with a depth of information without consuming significant bandwidth and computing resources.
The PACS server 110 serves as a traffic controller for accessing medical images from the archive database 140 and providing the images to the viewer device 115 for display to a user. The PACS database 105 includes a memory, for example, a non-transitory, computer-readable medium, storing medical information such as medical studies, metadata, images, reports, or a combination thereof. In some embodiments, the PACS database 105 comprises a plurality of databases. Also, in some embodiments, the PACS database 105 is included in the PACS server 110. Although not illustrated in
The archive database 140 is a medical image repository. For example, the archive database 140 may be a vendor neutral archive (VNA) or another suitable storage facility that stores medical image data generated by one or more different types of imaging modalities. The archive database 140 may comprise a single database system or a plurality of archive systems, which may be geographically dispersed. For example, one or more archive databases 140 may be used to populate the PACS database 105 whether the archive databases 140 are disparate systems or part of a business continuity or failover system. For example, in some embodiments, the archive database 140 includes geographically dispersed archives, wherein each archive contains a portion of the overall set of studies whether individually or in a partially replicated or redundant manner.
The archive database 140 stores one or more instances associated with one or more image studies. In general, an “instance” includes an object, such as an image, associated with an image study. For example, a still frame, a multi-frame sequence, or a movie generated as part of an image study may be referred to as an instance. Such image data may be formatted, stored, and transferred according to the digital imaging and communications in medicine (DICOM) format. Instances may also include non-image data, such as reports, scanned documents, or the like, that are encapsulated in a DICOM format or object. There are many types of DICOM objects, for example, images, reports, key image selections, key object selections, segmentation objects, registration objects, presentation states (typically comprising measurements or other annotations of images), and the like. Cross enterprise document sharing (XDS) objects or documents or other data formats or communication protocols may also be considered instances. XDS is an alternate storage mechanism that may be used in parallel to DICOM, such as for non-image documents and visible light images, such as skin photos or microscope images. Accordingly, the term “instance,” as used in the present application, includes all objects associated with a particular image study regardless of whether an object includes image data (pixel or voxel data).
While a one-to-one configuration of the PACS server 110 and the archive database 140 may be used as illustrated in
The PACS server 110 includes a plurality of electrical and electronic components that provide power, operational control, and protection of the components within the PACS server 110. For example, as illustrated in
The electronic processor 125 may be a microprocessor, an application-specific integrated circuit (ASIC), or other suitable electronic device. The memory 130 includes non-transitory computer-readable medium, such as read-only memory (ROM), random access memory (RAM) (for example, dynamic RAM (DRAM), synchronous DRAM (SDRAM), and the like, electrically erasable programmable read-only memory (EEPROM), flash memory, a hard disk, a secure digital (SD) card, other suitable memory devices, or a combination thereof. The electronic processor 125 accesses and executes computer-readable instructions (“software”) stored in the memory 130. The software may include firmware, one or more applications, program data, filters, rules, one or more program modules, and other executable instructions. For example, the software may include instructions and associated data for performing a set of functions, including the methods described herein.
The communication interfaces 135 allow the PACS server 110 to communicate with devices external to the PACS server 110. For example, as illustrated in
The viewer device 115 may be a desktop computer, a laptop computer, a workstation or terminal, a smartphone, a tablet computer, a smart television, a smart wearable, and the like. The viewer device 115 may include similar components as the PACS server 110. For example, as illustrated in
As illustrated in
The viewer device 115 allows users to access images stored in the PACS database 105 or the archive database 140 through the PACS server 110 over a network connection, such as a webpage (image viewer) accessible over the Internet through a browser application 157. For example, the viewer device 115 may access a web server that authenticates users accessing the PACS server 110 over the Internet through the browser application 157 and, when authenticated, allows users to access and view images stored in the archive database 140. Alternatively or in addition, the viewer device 115 may store a dedicated image viewer application for communicating with the PACS database 105, the PACS server 110, the archive database 140, or a combination thereof for accessing images. A user may select an image study to access through a worklist displayed by the image viewer and may generate a report for an image study, such as a structured report that may be completed using text-to-speech technology, using the image viewer.
As noted above, the PACS database 105 may be initially populated with metadata using an instance level or “full folder on the fly” (FFOF) method.
The method 200 is described as being performed by the PACS server 110 (the electronic processor 125 executing instructions stored in non-transitory computer readable medium, such as the memory 130). However, it should be understood that the method 200 (or portions thereof) may be performed by other devices (the archive database 140, the viewer device 115) or distributed among multiple servers.
As illustrated in
In some embodiments, as illustrated in
In some embodiments, the retrieved metadata also includes metadata regarding reports (DICOM SR objects, DICOM PDF objects, or DICOM CAD SR objects), XDS objects or documents, or other types of data. For example, for XDS objects or documents, the PACS server 110 may be configured to retrieve instance level metadata from one or a mix of a document registry and repository actors or devices. Given the size of such objects (non-image data objects), the PACS server 110 may retrieve both metadata for these objects as well as the objects themselves. For example, reports for any image studies represented by retrieved metadata may also be retrieved or pushed to the PACS server 110, such as from the archive database 140, an electronic medical record (EMR) of the patient, a radiologist information system (RIS), a report repository or the like. Such reports may be communicated using HL7 messaging (for example, medical document management (MDM) messages, observation result (ORU) messages, or the like). In some embodiments, reports that do not have existing image study entries in the archive database 140 may also be retrieved and used to populate the PACS database 105.
The instance level metadata includes pointers to image data stored in the archive database 140 (SOP instance unique identifier (UID)) as well as other information. This other information includes but is not limited to instance or slice number, photometric interpretation (monochromel, RGB, YUV, or the like), image resolution, image or slice position (three-dimensional vector information), phase, laterality (right or left), body part examined, contrast use, time delay from injection, cardiac cycle phasing, time delay from an excitation event, orientation such as axial, sagittal coronal, orientation via a position vector, MR acquisition sequence information, PET or NM attenuation correction type, laterality, view or orientation, angulation of the detector or x-ray geometry, ECG gating, respiratory gating, ultrasound acquisition type, presence of Key object selection, presentation state references, segmentation object references, registration object references DICOM SR object references, or the like. This information is typically used by an image viewer to organize and split the instance data into logical or physiological groupings presented via hanging protocols or default display layouts. As noted above, each image study may include as few as a single instance to many thousands of instances (images). Accordingly, the amount of metadata retrieved for a patient may include any amount of data.
Instance level metadata may optionally include series level metadata. For example, each image study includes one or more series, and each series includes one or more instances. A series may include a set of instances (images) generated as part of an image acquisition during the image study, such as images taken from a particular point of view or using a particular imaging procedure or technology. Similarly, a series may include a set of instances (images) generated from other images generated as part of an image study. For example, image data generated as part of an image study may be reformatted to change the view or other parameter of the images and these reformatted images may be stored as a separate series for the image study. Accordingly, when retrieving instance level metadata, series level metadata may be automatically retrieved as part of the instance level metadata. However, in other embodiments, series level metadata may be separately queried and retrieved. Series level metadata may include a series description, a series instance UID, contrast use information (multiple potential fields for type, injection or ingestion site, volume, and the like), modality (for example, CR, DX, MR, CT, and the like), acquisition sequence and protocol information, and the like.
Returning to
In other words, the metadata populated in the PACS database 105 allows instances in each prior or comparison image study for a patient to be characterized without the need to transfer the entire bulk of the images for all of the studies from the archive database 140. For example, as illustrated in
In particular, the metadata may be used to group images (into series or subseries), arrange and cache images in an optimal order, and navigate through images in an ad hoc manner. For example, the metadata allows the sequencing of the retrieval of the images to be optimized to properly hang or display the images filling the image screen layout and thumbnail images initially and fill in the balance of the study behind the scenes. In addition, having the series and instance level metadata, a medical practitioner can rapidly navigate a study in an ad hoc manner (based on sequential order, orientation, or other factors) with instance retrievals being guided by actions of the medical practitioner and not merely based on a bulk study retrieval of all images which more likely than not is in a non-optimum sequence for the practitioner's current viewing.
For example, based on the metadata, images can be grouped into series (including sub-series) and represented as thumbnails displayed within a pick list as illustrated in
In addition, when a user requests a particular instance (an image) based on the displayed information regarding the other studies, the pointer stored in the PACS database 105 allows the viewer device 115 to quickly access the requested instance. When the archive database 140 includes geographically dispersed archives storing duplicates of image data, the viewer device 115 may be configured to select the archive to retrieve image data from, such as from the closest or fastest archive. Similarly, in the event of a failure of a particular archive, the viewer device 115 may be configured to choose an alternate source for image data. It should be understood that, in some embodiments, the viewer device 115 receives image data from the archive database 140 through the PACS server 110. However, in other embodiments, the viewer device 115 receives image data from the archive database 140 (without interaction from the PACS server 110 other than providing the location (pointer) of the requested image).
As noted above, the PACS server 110 may implement one or more query rules 137 to determine what metadata to retrieve. In some embodiments, the query rules 137 may specify that metadata for all existing image studies for a patient should be retrieved from the archive database 140 or other medical information sources. As noted above, by retrieving instance level metadata but not the bulk image data for each instance, in many situations metadata can be retrieved and populated in the PACS database 105 for all image studies associated with a particular patient. However, in other embodiments, the query rules 137 may specify that only particular metadata is retrieved and used to populate the PACS database 105. For example, the rules may consider the trigger event, available image studies, a number of archive databases, or the like to determine the metadata to retrieve. In some embodiments, these rules are configurable to set particular rules for particular users (reading physician) or groups of users, patients, image study types, or the like.
Alternatively or in addition, the PACS server 110 may use a cognitive approach (using machine learning and various artificial intelligence techniques) to automatically learn what metadata to retrieve. For example, the PACS server 110 may record and learn what prior or comparison studies or instances users or groups of users request for particular types of image studies and use this training information to automatically determine query rules (a model) for retrieving metadata from the archive database 140. In particular, the PACS server 110 may track the types of studies accessed by a user as comparison studies, parts of anatomy imaged in studies assessed for comparison studies by a user, a role of a user performing a comparison study, findings included in reports for image studies, or the like. For example, the system 100 may use machine learning to build complex query rules that may be applicable to multi-site systems or cross-enterprise queries. Machine learning generally refers to the ability of a computer program to learn without being explicitly programmed. In some embodiments, a computer program is configured to construct a model (for example, one or more algorithms) based on example inputs. Supervised learning involves presenting a computer program with example inputs and their desired (for example, actual) outputs. The computer program is configured to learn a general rule (for example, a model) that maps the inputs to the outputs. The computer program may be configured to perform deep machine learning using various types of methods and mechanisms. For example, the computer program may perform deep machine learning using decision tree learning, association rule learning, artificial neural networks, inductive logic programming, support vector machines, clustering, Bayesian networks, reinforcement learning, representation learning, similarity and metric learning, sparse dictionary learning, and genetic algorithms. Using all of these approaches, a computer program may ingest, parse, and understand data and progressively refine models for data analytics.
Thus, embodiments described herein provide methods and systems for populating a PACS server with metadata, down to the instance level, quickly and efficiently, which allows for efficient organization and display of prior or comparison image studies without the need to prefetch the entire bulk image data for a study.
Various features and advantages are set forth in the following claims.
