Patent Application
20250157639
The present application claims priority under 35 U.S.C. § 119 to European Patent Application No. 23210160.0, filed Nov. 15, 2023, the entire contents of which is incorporated herein by reference.
One or more example embodiments relates to a technique for providing a control signal indicative of a workflow for performing medical imaging post-processing, image analysis and subsequent diagnosis via a plurality of radiology devices (in particular comprising medical scanners) out of a fleet of radiology devices. The technique comprises, in particular, a method, a computing device, a system comprising the computing device, a computer program product, and a computer-readable storage medium.
Independent of the grammatical term usage, individuals with male, female or other gender identities are included within the term.
Running a radiology department or independent radiology company is a challenging task. Usually, a “fleet” of several imaging systems including computed tomography (CT), mCT (molecular CT), magnetic resonance (MR), mMR (molecular MR) and X-ray devices, SPECT, PET are used for performing the medical imaging. Moreover, Digital Imaging and Communications in Medicine (DICOM)-routers, advanced visualization workplaces, Artificial Intelligence (AI) algorithms and reading & reporting software needs to be optimally set up to support the staff in performing the diagnostic task. Procedure throughput, patient experience (and/or patient satisfaction), and high-quality results for referrers are among the most common optimization criteria. Referrers of the radiology entity decide to send and possibly resend patients. The radiologist must deliver high-quality results in a timely manner, e.g., to optimize productivity, asset usage and also to get renewed referrals.
Conventionally, radiology operations require massive site, user, and indication specific configurations. Numerous systems are conventionally optimized individually and independently. The radiology staff (e.g., imaging technologist, briefly: “tech”, radiologist, and/or IT staff) conventionally collect a patient's data from different systems and select imaging protocols, configure subsequent steps such as reading tools, view and report templates. This conventional approach is time-consuming and error prone. It often does not result in a patient-specific optimal procedure optimized to the patient's clinical situation and suitable to answer the diagnostic task that the referrer ordered and/or for which medical imaging was requested.
As a result, imaging procedures are conventionally planned and executed based on data attributes which seem symptomatic for a certain action but are not connected to the overall root cause or goal. It is often unknown, why the scan needs to be done in the first place and which goal is to be achieved by the examination. The local rule set implements collateral information around the indication, reason for exam and expected outcome. This is not necessarily optimal for the exam result. An example is to perform a Lung Computer-Aided Detection (briefly: “Lung CAD”) by default if the technical expert (in short tech) at the scanner made sure that the DICOM-series description contains the substring “LCAD”. Thus, a cost and time intensive procedure would be carried out solely upon the received request and not by taking into account further requirements. Whilst this is one typical example, the complete automation with the radiology department mostly or completely relies on such rules, which need to be independently configured in the various radiology devices.
The conventional local process rule sets do not optimize towards the capabilities of the imaging devices available. Resources are conventionally not used optimally (e.g., optimally, the faster CT scanner should be used on the patient with arrythmias and configured to prioritize speed over a slower dual energy scan mode).
Furthermore, local (and/or manual) configurations of e.g., workflows, tools, and/or reports are error-prone and easily break. Moreover, conventional radiology operations usually involve high maintenance efforts by techs and secondary personnel to deal with these process shortcomings.
“Fleet management” conventionally focuses on firstly collecting information about the operations performed on scanners and secondly uploading and downloading scan protocols to the scanner fleet. The downloading of scan protocols is conventionally limited to identical (or rather similar) scanner models for all parameters (e.g., with technical parameters like slices, magnetic field, gradient strength rotation time and software, e.g., comprising licensing, and in particular the same scanning modality). Only the structure of the protocol names and output data structure to DICOM are conventionally communicated across the fleet. The communication of the output structure is conventionally used to standardize the output of the scanners. For AI/Advanced Visualization (AI/AV) stations, there is no comprehensive management solution on the market. Moreover, for operators of a fleet of scanners using multiple Picture Archiving and Communication Systems (PACSs) at different sites, there is no comprehensive fleet management.
In addition, there are scanners with an “Exam Companion” software guiding the technologist (“tech”) by asking questions about the patient, e.g., “Are there any bypasses?”. The conventional Exam Companion software adapts the scan protocol for the scanned patient based on the answers. This helps to translate a patient condition or goals of the examination into a technical parametrization of the scanner. The processing of the question-answer pairs is conventionally a manual and error-prone process. It conventionally requires that the radiology staff finds the respective information about the patient in prior documents or by asking the patient and enters it manually and correctly.
One or more example embodiments provides a solution for automatizing an optimal parametrization of the radiology devices involved in the diagnostic task comprising medical scanners (in particular comprising multiple scanning modalities), AI functions (e.g., as Software as a Medical Device, SaMD, as defined by the International Medical Device Regulators Forum, IMDRF), advanced visualization (AV), and/or PACS and/or report configurations and templates. The optimal parameterization of the radiology devices comprises optimally selecting a scanning protocol, and/or scan parameters, based on a request by a referrer. Alternatively or in addition, there is a need for a technique for improving the quality of requested medical imaging data subject to conditions on a fleet of medical scanners, AI/AV devices, workplaces, in particular in view of their availability and/or technical specifications. Further alternatively or in addition, it is an object of the invention to minimize errors and avoid wear and tear, and/or increase maintenance intervals for the devices (e.g., comprising medical scanners) within a fleet of radiology devices.
These are achieved by a method for providing a control signal indicative of a workflow for performing medical imaging (and optionally post-processing) via radiology devices (in particular comprising medical scanners) out of a fleet of radiology devices, by a computing device, by a system, by a computer program (and/or computer program product) and by a computer-readable storage medium according to the appended independent claims. Advantageous aspects, features and embodiments are described in the dependent claims and in the following description together with advantages.
In the following, the solution according to one or more example embodiments is described with respect to the claimed method as well as with respect to the claimed computing device. Features, advantages or alternative embodiments herein can be assigned to the other claimed objects (e.g., the system, the computer program or a computer program product), and vice versa. In other words, claims for the computing device, and/or the system, can be improved with features described or claimed in the context of the method. In this case, the functional features of the method are embodied by structural units of the computing device (and/or the system), and vice versa, respectively.
As to a method aspect, a (in particular computer-implemented) method for providing a control signal indicative of a workflow for performing medical imaging via a medical scanner out of a fleet of radiology devices (in particular comprising medical scanners) is provided. The method comprises a step of receiving first input data in relation to a request for medical imaging of a patient. The method further comprises a step of receiving second input data indicative of an availability of at least one radiology device (in particular at least one medical scanner) within a fleet of radiology devices. The method further comprises a step of processing the received first input data and the received second input data. Processing comprises selecting at least one available radiology device (in particular medical scanner) out of the fleet. In case more than one available radiology device (in particular medical scanner) is selected, it might be defined in a control signal, in which time sequence the radiology devices (in particular medical scanners) are to be applied or used. Processing is performed via a generative artificial intelligence (e.g., comprising a neural network, in particular comprising a transformer architecture), and/or a large language model (LLM), for creating a workflow of the medical imaging according to the request. The method still further comprises a step of providing the control signal indicative of the workflow for performing the medical imaging via the selected radiology device (in particular the selected medical scanner).
A radiology device may comprise a computing device used within a workflow of medical imaging, post-processing and medical imaging-based diagnosis and treatment. Alternatively or in addition, the radiology device may comprise any device (medical or non-medical, like e.g. a DICOM router) within a radiology department involved in the workflow of medical imaging, post-processing and medical imaging-based diagnosis and treatment.
The fleet of radiology devices may, e.g., comprise one or more medical or non-medical devices, one or more medical scanners, one or more DICOM-routers, and/or one or more devices configured for performing medical imaging post-processing, and/or for performing AI/AV functionality.
The fleet of radiology devices may in particular comprise a fleet of medical scanners (optionally: of different type).
By the inventive technique, a balance of workload of the radiology devices (in particular comprising the medical scanner), and in particular of the medical scanners within the fleet of radiology devices, AI calculations, AV and/or reading workplaces, may be significantly improved. Alternatively or in addition, a time period from the request to performing the medical imaging (also denoted as exam and/or radiology procedure) may be improved. This may in particular be beneficial for urgent cases and/or emergencies, e.g., after an accident and/or in case of a suspected severe illness requiring treatment in a timely manner. Further alternatively or in addition, a quality of the requested medical imaging data (also denoted as medical images) may be improved, e.g., by selecting the most suitable medical scanner (in particular including its availability, its technical specification, equipment and/or technical capabilities), the needed AI/AV evaluations, the best visualization (e.g., comprising hanging protocols), the most suitable report template for the specific request within the fleet and/or by optimizing the workflow of the selected radiology device (e.g., the selected medical scanner). Still further alternatively or in addition, by the inventive technique (e.g., transfer-related and/or human) errors (e.g., due to a medical image not corresponding to the request), premature deterioration, and/or maintenance instances (and/or costs, e.g., due to an erroneous selection of a configuration of the medical scanner) may be avoided (and/or at least mitigated).
The fleet of radiology devices may comprise a plurality of radiology devices (in particular jointly) operated by a fleet operator. The fleet operator may comprise a medical institution (in particular a hospital, and/or a collective of hospitals in a predetermined local area), a radiology department, and/or a radiological practice.
The second input data indicative of the availability of the at least one radiology device (in particular the at least one medical scanner) may be received passively (e.g., without request by the generative AI and/or a computing device comprising the generative AI), and/or actively (e.g., responsive to a request by the generative AI, and/or via a measurement initiated by the generative AI, and/or the computing device comprising the generative AI). Alternatively or in addition, the second input data indicative of the availability of the at least one radiology device (in particular the at least one medical scanner) may be received based on a push mechanism, a pull mechanism, or a combination thereof. E.g., the at least one radiology device (in particular the at least one medical scanner) may provide the second input data indicative of its availability in case of an event which changes the availability (e.g., if a medical scanner is put to immediate use due to a medical emergency). The pull mechanism may comprise a request for receiving the second input data indicative of the availability of the at least one radiology device (in particular the at least one medical scanner), e.g., on a periodic basis, if the first input data in relation to the request for medical imaging of the patient is received (e.g., at a computing device comprising the generative AI), and/or if the generative AI for creating the workflow of the medical imaging identifies the at least one radiology device (in particular the at least one medical scanner) as (in particular ideally) suitable (e.g., due to its technical specification and/or equipment) for the request for medical imaging of the patient.
For medical scanners, availability is a relevant optimization criterion. The subsequent paragraphs may also apply to workplaces and/or optimization of the concurrent use and/or availability of licenses.
Selecting at least one available medical scanner (e.g., out of the fleet and/or out of a plurality of medical scanners) may comprise to (in particular automatically and/or digitally) match and/or allocate an available medical scanner to a patient to be examined. Alternatively or in addition, the step of selecting may be executed algorithmically.
The step of selecting may process a time condition and/or a location condition, so that the match and/or allocation “medical scanner-patient” considers a time phase, in which the medical scanner is available and/or the patient is to be examined, and/or considers a location (e.g., by processing location-based information, e.g., a GPS signal and/or indoor navigation such as ultra-wideband, UWB, tags) of the medical scanner and/or the patient.
By the inventive technique, improved image quality in view of the received request (leading to an improved diagnostics) and an improved patient outcome may be enabled.
The first input data may comprise a reason for the request and/or an organ and/or anatomical structure to be imaged. Alternatively or in addition, the first input data may comprise data related to the health of the patient, e.g., age, height, weight, and/or recently measured vital signs (e.g., blood pressure, and/or heart rate). Further alternatively or in addition, the first input data may comprise laboratory (briefly: lab) values, former surgeries (and/or an amputation) and/or a presence of implants (e.g., a pacemaker, stent, artificial joint, and/or dental implant). Alternatively or in addition the first input data may comprise prior exams and/or an amount of prior exams.
The request may comprise a preference, and/or a need for, a Predetermined medical imaging modality, and/or evaluations to be performed.
The patient may be a human (and/or person) or an animal.
The second input data may comprise and indication at what times a medical scanner, and/or another time-limited radiology device, are available or not available or available with a certain (e.g. limited) functionality. This may also incorporate availability of needed human experts, like techs or radiologists. E.g., musco-skeletal experts are usually only available at office times.
The availability of the medical scanner (and/or of any medical scanner within the fleet) may comprise that the medical scanner is technically in good order, no previously scheduled medical imaging exists for the same time slot, and/or (in particular specifically trained) operating personal (and/or a technician, briefly: tech) is scheduled to be present.
Alternatively or in addition, the radiology device (in particular the medical scanner) may not be available in case of a technical fault, missing equipment, a (e.g., scheduled) maintenance, no (in particular specifically trained) operating personal is available, and/or the radiology device (in particular the medical scanner) is scheduled for use for other (e.g. urgent) requests, in particular in relation to another patient.
Any one of the first input data and the second input data may be received as raw data (not pre-processed, e.g., stemming from different sources) and/or data in predetermined different formats, e.g., text data and/or in a different syntax. Alternatively or in addition, the first and/or second input data may be received in a pre-processed form, e.g. in order to harmonize and unify the different data sources and to transform them in on common form, which is processable by the generative artificial intelligence.
Alternatively or in addition, any one of the first input data and the second input data may be received in a different format (e.g., a format conventionally used for storing and transmitting the respective data). The (e.g., first and/or second) input data may be pre-processed to be available as pre-processed data (and/or data in a predetermined format, e.g., text data) for the (in particular further) processing.
The medical scanner (briefly also: scanner) may also be denoted as medical imaging device. Alternatively or in addition, the medical scanner may comprise a device for computed tomography (CT), a device for magnetic resonance tomography (MRT, also denoted as magnetic resonance imaging, MRI, or briefly also: MR), a device for X-ray imaging (also denoted as radiography), an ultrasound (US) device, a device for single-photon emission tomography (SPECT) and/or a device for positron emission tomography (PET).
Radiology devices (in particular medical devices) in radiology may alternatively or in addition comprise AI functions (e.g., as software, in particular implemented on dedicated hardware, and/or cloud-implemented) that can be used, e.g., for automatic identification of lung nodules (LungCAD), automatic identification of pneumothorax, and/or automatic assessment of cortical thickness in the brain.
Alternatively or in addition, radiology device (in particular medical) devices may comprise advanced visualization systems that can perform automatic actions, like reconstructions, and/or provide and/or offer) interactive functions, e.g., to assess arteriosclerosis. The interactive functions may be provided (and/or offered) in a performance optimized manner to the user (e.g., radiology staff), if suitable pre-processing has been performed automatically on the advanced visualization systems.
The timely and adequate triggering of pre-processing of the advanced visualization systems may comprise functions that can be automatically optimized within the scope of example embodiments.
Further alternatively or in addition, diagnostic reading software and/or workplaces (e.g., comprising hardware and/or software) may be comprised in the fleet of radiology devices (in particular medical devices) that can be optimized within the scope of example embodiments, e.g., with respect the hanging protocols.
Still further alternatively or in addition, radiology devices (e.g., which in most legislations are not a classified as medical devices) may comprise DICOM-routers and/or reporting systems. The behavior and/or configuration of the DICOM-router and/or reporting system can also be accordingly optimized (e.g., which data needs to be routed to which workplace or which reporting template to use, in particular for post-processing of the acquired medical imaging data).
The fleet of radiology devices may comprise at least two different medical scanners. Alternatively or in addition, the fleet of radiology devices (in particular comprising medical scanners) may comprise at least two medical scanners using different medical imaging modalities (e.g., at least CT and MRT).
Two medical scanners using the same medical imaging modality may differ in their technical specification and/or equipment (e.g., in terms of a magnetic field and/or field gradient of an MRT scanner, and/or a capability of performing a particular functionality, e.g., like dual energy scanning of a CT scanner) and/or equipment (e.g., an existence of coils).
Alternatively or in addition, radiology devices may differ by the services (e.g., post-processing, reconstruction, and/or AI/AV functionality) offered, licenses available, processing speed and/or location.
The medical scanner (and/or the medical scanners within the fleet of radiology devices) may be associated with (in particular a radiology department of) a medical facility, e.g., a hospital (and/or an association of hospitals, e.g., in a town, district or otherwise predefined area) or an out-patient facility such as a doctor's office (and/or an association of doctor's offices).
Receiving the first and/or second input data may be executed via a computer in an automatic manner via an electronic interface. Receiving may be construed as digitally receiving.
Providing the control signal relates to providing a preferably interim, result. The result comprising the control signal may be used to control the medical scanner for medical imaging, in particular during an acquisition phase (and/or an application phase). Providing is to be constructed as digitally providing, which may be performed via an interface, in particular a control signal providing interface.
The processing may be performed using a large language model (LLM), deep learning (DL), a foundation model, a generative model, and/or a generative intelligence, in particular a generative artificial intelligence (AI).
The generative AI (briefly also: GenAI) may comprise a neural network (NN), e.g., comprising a transformer architecture.
Generative AI may be capable of generating text, images, and/or other media, using generative models. Alternatively or in addition, the generative AI may learn the patterns and structure of their input training data and then generate new data that has similar characteristics.
The generate AI may comprise deep learning (DL), and/or unsupervised and/or preferably self-supervised learning.
The generative AI may be unimodal or multimodal. E.g., if all received (e.g., first and second) input data, and optionally technical specifications and/or generally applicable instructions, are pre-processed and normalized to a specific (in particular text) format, the generative AI may be unimodal. Alternatively, the generative AI being multimodal may comprise the generative AI being trained on a variety of (in particular text) formats, e.g., natural language text and/or programming language text.
The generative AI may comprise a foundation model (also called base model). The foundation model may comprise a large machine learning (ML) model trained on a vast quantity of data at scale (e.g., by self-supervised learning and/or semi-supervised learning) such that it can be adapted to a wide range of (e.g., downstream and/or subsequent) tasks.
Examples of foundation models comprise pre-trained language models (LMs) including bidirectional encoder representation from transformers (BERT), as described by J. Devlin et al. in arxiv:1810.04805 [cs.CL] [1], which is incorporated herein by reference. Further examples of foundational models comprise GPT foundation models, e.g., “GPT-n” series.
Alternatively or in addition, the generative AI may comprise an attention-based NN, as described by A. Vaswani et al in “Attention is all you need”, arXiv:1706.03762 [cs.CL] [2], which is incorporated herein by reference.
Further alternatively or in addition, the generative AI may comprise a transformer, and/or a NN with transformer architecture. A brief overview over transformer models is provided on https://huggingface.co/docs/transformers/model_summary [3], which is incorporated herein by reference.
The transformer architecture may comprise a deep learning (DL) architecture comprising a, in particular parallel and/or multi-head, attention mechanism.
The transformer architecture may comprise a bidirectional autoregressive transformer (BART), e.g., as described by Lewis et al. in arXiv:1910.13461 [ ], which is incorporated herein by reference. Alternatively or in addition the transformer architecture may comprise at least one decoder, and preferably at least one encoder. Further alternatively or in addition, the transformer architecture may comprise BERT, as also described, e.g., by Rogers et al. in arXiv:2002.12327 [5], which is incorporated herein by reference.
The transformer architecture may be trained using historical (also denoted as real) and/or synthetic sets of input data (in particular comprising the first input data in relation to the request for medical imaging and the second input data indicative of the availability of one or more medical scanners within the fleet) with already associated or matched workflows, for which the association or matching has been found to comply with pre-definable quality criteria as the ground truth.
The neural network (NN) comprising the transformer architecture may comprise an LLM.
The NN comprising the transformer architecture may perform natural language processing (NLP) of medical documents. E.g., the processing of the first input data may comprise extracting relevant information for the medical imaging from the received (and/or pre-processed) first input data relating to the patient.
Alternatively or in addition, the generative AI may comprise a Generative Adversarial Network (GAN), in particular a Generative Adversarial Transformer, as described by D. A. Hudson and C. L. Zitnick in arXiv:2103.01209 [cs.CV] [6], which is incorporated herein by reference.
Any generative AI may face the challenge of achieving training parallelism (e.g., as compared to sequential training), low-cost inference, and/or strong performance. The challenge of achieving all three goals simultaneously may be referred to as “impossible triangle” (as, e.g., illustrated in FIG. 2 in Y. Sun et al. in arXiv:2307.08621 [cs.CL], [7]), with linear transformers achieving training parallelism and low-cost inference, but not string inference, with recurrent neural networks (RNNs) achieving low-cost inference and strong performance, but failing at training parallelism, and with transformers achieving training parallelism and strong performance, but failing at low-cost inference.
Still further alternatively or in addition, the generative AI may comprise a retentive network (RetNet), as described by Y. Sun et al. in arXiv:2307.08621 [cs.CL] [7] and by S. Chandra in https://medium.com/ai-fusion-labs/retenBve-networks-retnetexplained-the-much-awaited-transformers-killer-is-here-6c17e3e8add8 [8], both of which are incorporated herein by reference.
The RetNet may, e.g., have a better language modelling performance than any (linear and/or non-linear) transformer or RNN, achieve the better language modelling performance with (e.g., 3.4×) lower memory consumption, (e.g., 8.4×) higher throughput, and (e.g., 15.6×) lower latency.
In an embodiment of the generative AI, a local attention mechanism may be comprised. The local attention mechanism may enable a (e.g., very) large trained NN to operate focused locally. Alternatively or in addition, the generative AI may be post-trained (e.g., downstream) and/or be (e.g., continuously) fine-tuned.
In an embodiment, a radiology device (and/or computing device) configured for performing the method may be trained and tested by the manufacturer then shipped to its deployment location (e.g., radiology department). The radiology device (and/or computing device) need not be configured to learn (and/or perform post-training) at the deployment site.
In a further embodiment, the radiology device (and/or computing device) may be configured to be continuously fine-tuned in operation. E.g., by user feedback and/or by observing workflows or/or reports. E.g., if users consistently override the automatically generated workflow, the previously trained radiology device (and/or computing device) may be optimized.
The workflow (also denoted a schedule, and/or as procedure workflow) may comprise, at the scanner, a selection of scan parameters (also: scanning parameters; briefly: parameters), a configuration of the medical scanner, a scanning protocol (briefly also denoted as protocol), a sequence plan, and/or a scanning sequence to be executed on the medical scanner. Thus, the workflow may comprise technical features (instructions and/or parameters and/or other features) for performing the medical imaging.
Alternatively or in addition, the workflow may comprise an AI functionality, which is configured to perform steps of a diagnostic process, and/or which may be configured to not perform to save processing time, and/or which may be configured to save time of the medical staff to assess the data created (e.g., what is created needs to be analyzed).
Further alternatively or in addition, the workflow may comprise AV functions, in particular to pre-process the acquired medical imaging data.
Still further alternatively or in addition, the workflow may comprise visualizing data within the reading process, and/or selecting which data to (e.g., simultaneously) display out of the available prior data.
Still further alternatively or in addition, the workflow may comprise providing a report template to use, and/or which report to fill based on guidelines provided.
The radiology devices may be selected, and the workflow created, in order to optimize the requested medical imaging data in terms of quality, appropriateness for the reason, and/or timeliness of the acquisition, in particular in view of all requests for all patients within a particular time phase for the radiology devices of the fleet and/or available personnel. Thus, selecting a radiology device (e.g., a medical scanner) out of the fleet may be executed on a superordinate level for all radiology devices (e.g., of a particular type, e.g., all medical scanners) of the fleet and for all requests in common, e.g., and not only for one particular radiology device. E.g., in case of suspected injury after an accident as the reason, the nearest suitable medical scanner may be chosen in order to avoid further transports, mechanical shocks or vibrations, and/or time delays.
The control signal may be configured for enabling the selected medical scanner to perform the (in particular complete) workflow. Alternatively or in addition, the control signal may be provided in a format, which the medical scanner is able to import.
At least the steps of processing the received first and second input data and providing the control signal may be repeatedly performed, e.g., if an availability of the medical scanner changes, if the request is modified, and/or if further data in relation to the patient are received (e.g., lab data and/or previously acquired medical images, in particular comprising a preferred hanging, of the patient are received after the request has been received according to the first input data).
A hanging may refer to a hanging protocol, in particular according to the DICOM standard, and may comprise a spatial arrangement of multiple medical views and/or multiple medical images (e.g., various sectional views of an organ and/or anatomical structure, such as the head, and/or or views displaying selected anatomical structures of the organ and/or anatomical structure, such as soft tissue, e.g., brain, or bone structures) on a computer screen. A hanging (protocol) may refer to a set of viewing instructions determining the layout and display of the image(s). It allows the user to initiate the viewing of an image or study based on configurable attributes, like, e.g., the modality type, number of images, availability of comparison images and/or many other user defined criteria. Predefined Hanging Protocols allow standardized viewing settings for the specific image or exam type and thus improve quality and efficiency.
The method may further comprise a step of receiving a technical specification of the radiology device (e.g., the medical scanner), and preferably for any radiology device (e.g., any medical scanner) within the fleet of radiology device (e.g., comprising medical scanners). Optionally, the technical specification may comprise an existence and/or specification of, in particular detachable, equipment (e.g., one or more coils).
The technical specification may, e.g., comprise a size of an opening (e.g., for CT), a maximal weight capacity (e.g., of a patient table), an availability of a scanning protocol, and/or a rotation time (e.g., of the detector of the medical scanner).
Alternatively or in addition, the technical specification may comprise a magnet field strength and/or a gradient field strength (also: gradient strength) for MRT.
Further alternatively or in addition, the technical specification may comprise a capability of imaging (e.g., a number of) slices, a capability for dual energy medical imaging, a capability for photon counting, and/or a capability of physiological triggering.
The “physiological triggering” may refer to when to perform medical image acquisition relative to the cardiac cycle and/or respiration of the patient. E.g., cardiac triggering may be used in MRT and/or CT. Due to the fact that MRT is usually slower than CT, respiratory triggering and gating may alternatively or in addition be used there.
Gating may comprise that a measurement (and/or, in particular medical imaging, data acquisition) takes place and data is taken, sorted in or thrown away. Alternatively or in addition, triggering may comprise that (in particular medical imaging) data acquisition is performed (and/or happens) triggered by, e.g., the correct cardiac phase.
Still further alternatively or in addition, the technical specification may comprise a list of admissible scan parameters (e.g., comprising an interval and/or set of values per scan parameter), possible configurations of the medical scanner, available scanning protocols, available sequence plans, and/or available scanning sequences.
The (in particular detachable) equipment (also: tools) may comprise one or more coils (e.g., radio frequency, RF, coils for MRT).
The technical specification of any one of the further radiology devices (e.g., devices comprising AV functionality, AI functionality, and/or DICOM-routers) may comprise (in particular data) interfaces and/or software installed on the respective radiology device.
The technical specification may be received (e.g., only once) after deployment of the radiology device (e.g., the medical scanner). Alternatively or in addition, the technical specification may be updated after a maintenance and/or an upgrade of the radiology device, in particular the medical scanner (e.g., a software upgrade, and/or renewal and/or expansion of its equipment). The technical specification may be received continuously or repeatedly, in particular after deployment, and/or more than one time and/or may change over time.
There is provided a feedback channel for referring back signals from the radiology device (e.g., the medical scanner, in short also: scanner) to the computing device which is configured for providing the control signal for the radiology device (e.g., the medical scanner). With this, a closed loop control for controlling the radiology device (e.g., the medical scanner) out of the fleet of radiology devices (in particular comprising medical scanners) is provided. In particular, the second input data and/or optionally the technical specification of the radiology device (e.g., the medical scanner) may be fed back to the computing device, in particular during processing. This allows for dynamically adapting the control signal to the actual situation at the specific radiology device (e.g., scanner). E.g., if, for example, a workflow has been set up, represented in the control signal and an emergency case is to be examined on the particular scanner in question, which was assigned to a certain patient examination, a new scheduling and/or a new workflow has to be determined, taking into account the particular emergency situation and allocation (e.g. indicating that the original scanner is no longer available and another scanner needs to be assigned). Another example relates to a breakdown or failure of a particular medical scanner and transmission of this information back to the computing device in charge of providing the control signal.
Alternatively or in addition, the method may further comprise a step of receiving generally applicable instructions in relation to medical imaging by the fleet of medical scanners. The generally applicable instructions may in particular comprise one or more medical guidelines and/or one or more standard operation procedures (SOPs).
The generally applicable instructions may be fleet-specific (e.g., depending on the medical imaging modalities within the fleet) and/or specific to the operator of the fleet of medical scanners.
The received first input data in relation to the request for medical imaging of the patient may comprise a reason for requesting the medical imaging, in particular comprising a suspected medical condition and/or a suspected lesion.
The reason may also be denoted as medical question. Alternatively or in addition, the reason may comprise an event, e.g., the patient having experienced an accident.
The first input data, and/or the reason for the request, may comprise an indication of the patient's transportability (which may, e.g., be reduced in case of an expected concussion, severe injury and/or significant blood loss).
Alternatively or in addition, the received first input data in relation to the request for medical imaging of the patient may comprise one or more anatomical structures, and/or one or more organs, to be captured by the medical imaging.
The one or more anatomical structures (and/or organs) to be imaged may, e.g., comprise the head area (in particular comprising the brain, e.g., in case of a suspected aneurysm in that area), the chest (e.g., in particular the heart and/or the lung), and/or a joint area (e.g., at a knee and/or the hip).
Further alternatively or in addition, the received first input data in relation to the request for medical imaging of the patient may comprise information on the patient's general health state, in particular in view of the presence of a bypass, amputation, stent, pacemaker, and/or any (in particular further) implant.
The information on the patient's health state may be received or derived from an electronic health record (EHR) and/or electronic medical record (EMR), laboratory results and/or recorded vital signs.
Still further alternatively or in addition, the received first input data in relation to the request for medical imaging of the patient may comprise existing medical images of the patient (prior exams), in particular in relation to the request for which the medical scanner has been selected to provide the medical images in accordance to quality requirements.
The existing medical images may, e.g., be indicative of one or more requested views (and/or hangings) for the present medical imaging. By comparing the existing medical images with the present medical imaging, a lesion may be followed up, and/or a success of a surgical procedure may be controlled.
Alternatively or in addition, the existing medical images may be indicative of a position of a bypass, an implant (e.g., a stent and/or pacemaker), and/or an amputation.
At least a part of the first input data in relation to the request for medical imaging of the patient may be received via a user interface (UI), in particular a graphical user interface (GUI).
The request may be issued (and/or entered) by health care professional (e.g., a general practitioner, and/or a specialist such as an internist, a surgeon, an orthopedic, and/or an oncologist). The issuer of the request may also be denoted as referrer.
The request may, e.g., by received from a computer of the referrer. Alternatively or in addition, the UI (in particular the GUI) may be arranged at a location associated with the fleet of medical scanners.
The processing may comprise selecting at least one medical scanner (or several medical scanners to be operated in sequence) based on the technical specification of the medical scanner. Alternatively or in addition processing may comprise determining a set of scan parameters for the workflow.
The processing may take into account previous medical images (also denoted as prior exams) for patients with similar features, e.g., in view of equipment used, measurements performed, ranges of scan parameters used, and/or views selected.
The creating of the workflow may be further based on historical data. Alternatively or in addition, the generative AI (e.g., embodied by the NN, in particular comprising the transformer architecture) may be trained to select the medical scanner and the (in particular for the medical scanner allowable) scan parameters based on historical data.
The scan parameters may also be denoted as operating parameters (and/or denoted as scan settings, briefly also settings). Alternatively or in addition, the scan parameters may parameterize the workflow (e.g., a scan parameter may comprise a number of slices, in particular of a CT scan, and/or a scanning protocol, in particular of an MRT scan).
The processing may alternatively or in addition comprise selecting the medical scanner and/or determining (in particular optimizing) the set of scan parameters for the workflow according to (in particular constraints on the availability of) a time sequence of (e.g., previously created) workflows of the medical scanner (and/or of any one of the medical scanners, in particular considered a priori suitable for the request, within the fleet). Further alternatively or in addition, the processing may comprise selecting the medical scanner and/or determining (in particular optimizing) the set of scan parameters for the workflow according to a location constraint, e.g., prioritizing a short transfer way and/or a short transfer time for a patient with a medical emergency.
In some embodiments, the processing may comprise selecting the medical scanner, even if it was occupied by other workflows according to the received second input data, if the request for medical imaging comprises a priority, which is higher than a priority of the other workflows (and/or if the other workflows have not been assigned a priority).
The generative AI may comprise an NN comprising an attention mechanism.
Alternatively or in addition, the generative AI may comprise an NN comprising a transformer architecture.
Further alternatively or in addition, the generative AI may comprise a retentive network.
Still further alternatively or in addition, the generative AI may comprise a generative adversarial network (GAN), in particular a generative adversarial transformer.
Even further alternatively or in addition, the generative AI may comprise an LLM.
The generative AI may be trained (and/or pre-trained) using unsupervised, self-supervised, and/or semi-supervised learning.
The method may further comprise a step of pre-processing the received first input data in relation to the request for medical imaging of the patient. Alternatively or in addition, the method may further comprise a step of pre-processing the received second input data indicative of the availability of at least one medical scanner within the fleet. Further alternatively or in addition, the method may further comprise a step of pre-processing the received technical specification of the medical scanner. Still further alternatively or in addition, the method may further comprise a step of pre-processing the received generally applicable instructions.
The step of pre-processing may be performed after receiving the (e.g., first and/or second) input data (e.g., collectively, and/or after completing an iterative reception, in particular of the first input data), in particular by a computing device.
Alternatively or in addition, the first input data in relation to the request for medical imaging of the patient may be first (and/or individually) pre-processed before the receiving, in particular by a computing device, for the processing by the generative AI (e.g., comprising a NN).
Pre-processing may comprise structuring the (e.g., first and/or second) input data, the technical specification, and/or the generally applicable instructions, and/or converting the (e.g., first and/or second) input data, the technical specification, and/or the generally applicable instructions into pre-processed data and/or a common data format (in particular readable by the generative AI, such as a NN, performing the processing). Alternatively or in addition, structuring may comprise ordering, e.g., findings, by organ and/or anatomical structure.
Alternatively or in addition, pre-processing may comprise normalizing the (e.g., first and/or second) input data, the technical specification, and/or the generally applicable instructions, in particular to be available in common data format (e.g., from raw data). Alternatively or in addition, normalizing the (e.g., first) input data, and/or the generally applicable instructions, may comprise transforming natural language text (e.g., the text of the request, a medical guideline, and/or SOP) into text (and/or data) according to a predetermined ontology, e.g., the systematized nomenclature of medicine (SNOMED) in particular suitable to be processed by the generative AI.
Pre-processing may be performed using one or more further NNs (e.g., one per kind of input data, e.g., according to first input data or second input data, and/or within the first input data according to the type of data comprising text in a predetermined format and/or image information), e.g., comprising a transformer architecture. The (e.g., transformer) architecture of the further NNs for pre-processing may be independent (and/or different) from a (e.g., transformer) architecture of an NN for processing.
In an alternative embodiment, the (e.g., transformer) architecture of the NN for the processing and the further NNs for the pre-processing may (e.g., at least partially) be identical.
The method may further comprise a step of acquiring (in particular according to the request for medical imaging), by the selected medical scanner, medical imaging data based on the provided control signal.
The acquired medical imaging data may be further stored in a (e.g., Digital imaging and communications in medicine, DICOM, and/or picture archiving and communication systems, PACS) database.
Alternatively or in addition, the method may further comprise a step of post-processing the acquired medical imaging data.
Post-processing may comprise reconstructing a medical image from acquired medical imaging data. Alternatively or in addition, post-processing may comprise preparing (also denoted as pre-processing) acquired medical imaging data for further use, e.g., for segmentation, LungCAD, and/or diagnostic purposes.
Post-processing may comprise selecting one or more views and/or hangings of the views, e.g., according to the request comprised in the first input data.
Alternatively or in addition, post-processing may comprise using an AI/AV (advanced visualization) software on the acquired medical imaging data and/or analysing the acquired medical imaging data in view of the reason comprised in the request. E.g., a segmentation (e.g., of an organ and/or anatomical structure) may be performed, and/or an extension of a lesion may be (in particular automatically) measured.
The AI/AV software (e.g., comprising syngo.via, briefly also via, and/or AI Rad Companion, AIRC) may comprise an analysis module and/or analysing functionality. Alternatively or in addition, the AI/AV software may be configured for analysing, evaluating, visualizing interactively and/or visualizing automatically (e.g., the acquired) medical imaging data.
The AI/AV software may, e.g., be configured for 3D rendering of diffusion imaging and/or perfusion imaging in relation to the acquired medical imaging data.
By the AI/AV software, (e.g., in a first iterative step) a visualization may be provided to a user (e.g., radiology staff). The user may either confirm or reject the provided visualization. Alternatively or in addition, the user may provide instructions and/or indications for a modified visualization, which is to be provided by the AI/AV software (e.g., in a second iterative step).
Further alternatively or in addition, post-professing may comprise preparing and/or transmitting a (in particular textual) report on the medical imaging data and/or findings therefrom.
Post-processing may make use of one or more still further NNs, e.g., comprising a transformer architecture. The (e.g., transformer) architecture of the still further NNs (e.g., one per kind of output data) may be independent of the (e.g., transformer) architecture of the NN performing the processing and/or the (e.g., transformer) architecture of the one or more further NNs for the pre-processing.
Post-processing may be performed locally at the medical scanner, by which the medical imaging data were acquired, by a computing device (e.g., a workstation), at a computing device (e.g., a workstation) comprising a display, externally (e.g., at a central computing location), cloud-based, and/or in any combination thereof.
The method may further comprise a step of displaying the created workflow using a UI, in particular a GUI. Alternatively or in addition, the method may further comprise a step of displaying the acquired and/or post-processed medical image data (e.g., on a UI, in particular a GUI). In one embodiment, the displays (e.g., UI, in particular GUI) for the created workflow and the acquired and/or post-processed medical image data may be independent of each other.
By the display of the created workflow, and/or of the medical imaging data, a (in particular human) operator (and/or technician) of the medical scanner can verify the correctness of the created workflow and the thereupon acquired medical imaging data. Thereby, an adjustment and/or correction may be performed in a timely manner, if a deviation from the request is detected. Alternatively or in addition, scheduling another appointment for further medical imaging of the patient may be avoided.
The medical imaging may be performed via a medical imaging modality such as CT, MRI, X-ray imaging (also denoted as radiography), ultrasound/US, SPECT, and/or PET and/or for molecular appliances, like mCT, mMRI.
The fleet of medical scanners may comprise at least two medical scanners associated with different medical imaging modalities. The fleet may, e.g., comprise at least one CT scanner and at least one MRT scanner.
Depending on the reason for requesting the medical imaging, the processing by the generative AI (e.g., NN, in particular comprising the transformer architecture) may comprise selecting the medical imaging modality.
The method may be used for scheduling a plurality of patients across the fleet of medical scanners.
The inventive technique may be used to streamline and/or simplify all medical imaging requests and/or scheduling of the fleet of medical scanners. E.g., the combined schedule of the fleet may be iteratively filled responsive to the received requests for medical imaging.
As to a device aspect a computing device for providing a control signal indicative of a workflow for performing medical imaging via a medical scanner out of a fleet of radiology devices (in particular comprising medical scanners) is provided. The computing device comprises a first input data receiving interface configured for receiving first input data in relation to a request for medical imaging of a patient. The computing device further comprises a second input data receiving interface configured for receiving second input data indicative of an availability of at least one radiology device (in particular at least one medical scanner) within a fleet of radio devices. The computing device further comprises a processing unit configured for processing the received first input data and the received second input data. The processing comprises selecting at least one available radiology device (in particular medical scanner) out of the fleet. Processing is performed via a generative AI (e.g., an NN comprising a transformer architecture, and/or an LLM) for creating a workflow of the medical imaging according to the request. The computing device still further comprises a control signal providing interface configured for providing a control signal indicative of the workflow for performing the medical imaging via the selected radiology device (in particular the selected medical scanner).
Optionally, the computing device may comprise a technical specification receiving interface configured for receiving a technical specification of the medical scanner, preferably for any radio device (and/or particularly for any medical scanner) within the fleet of medical scanners. Optionally, the technical specification comprises an existence and/or specification of, in particular detachable, equipment (e.g., one or more coils).
Alternatively or in addition, the computing device may comprise a generally applicable instructions receiving interface configured for receiving generally applicable instructions in relation to medical imaging by the fleet of radiology device medical scanners, in particular comprising one or more medical guidelines and/or one or more SOPs.
Further alternatively or in addition, the computing device may comprise a pre-processing unit. The pre-processing unit may be configured for pre-processing the received first input data in relation to the request for medical imaging of the patient. Alternatively or in addition, the pre-processing unit may be configured for pre-processing the received second input data indicative of the availability of at least one medical scanner within the fleet. Further alternatively or in addition, the pre-processing unit may be configured for pre-processing the received technical specification of the medical scanner. Still further alternatively or in addition, the pre-processing unit may be configured for pre-processing the received generally applicable instructions.
Further alternatively or in addition, the computing device may comprise a medical imaging data acquisition interface configured for receiving medical imaging data acquired by the selected medical scanner based on the provided control signal.
Still further alternatively or in addition, the computing device may comprise a post-processing unit configured for post-processing the acquired medical imaging data.
Alternatively or in addition, the computing device may comprise a workflow display interface configured for displaying the created workflow using a UI (in particular a GUI). Further alternatively or in addition, the computing device may comprise a medical imaging data display interface configured for displaying the acquired and/or post-processed medical image data (e.g., on a UI, in particular a GUI). The displays (e.g., UI, in particular GUI) for the created workflow and the acquired and/or post-processed medical image data may be independent of each other.
The computing device may be configured to perform any one of the steps, or comprise any one of the features, described in the context of the method aspect.
As to a system aspect, a system for providing a control signal indicative of a workflow for performing medical imaging via a medical scanner out of a fleet of radiology devices (in particular comprising medical scanners) is provided. The system comprises a computing device according to the device aspect. The system further comprises a fleet of radiology devices (in particular comprising medical scanners). Each radiology device (in particular each medical scanner) within the fleet may comprise a control signal receiving interface configured for receiving a control signal from the control signal providing interface of the computing device.
The system may be cloud-based.
As to a further aspect, a computer program product is provided. The computer program product comprises program elements which induce a computing device to carry out the steps of the method for providing a control signal indicative of a workflow for performing medical imaging via a medical scanner out of a fleet of radiology devices (in particular comprising medical scanners), according to the method aspect, when the program elements are loaded into a memory of the computing device.
As to a still further aspect, a computer-readable medium on which program elements are stored is provided. The program elements can be read and executed by a computing device, in order to perform steps of the method for providing a control signal indicative of a workflow for performing medical imaging via a medical scanner out of a fleet of radiology devices (in particular comprising medical scanners), according to the method aspect, when the program elements are executed by the computing device.
The properties, features and advantages described above, as well as the manner they are achieved, become clearer and more understandable in the light of the following description and embodiments, which will be described in more detail in the context of the drawings.
This following description does not limit the invention on the contained embodiments. Same components or parts can be labeled with the same reference signs in different figures. In general, the figures are not for scale.
It shall be understood that an example embodiment can also be any combination of the dependent claims or above embodiments with the respective independent claim.
These and other aspects will be apparent from and elucidated with reference to the embodiments described hereinafter.
Any reference signs in the claims should not be construed as limiting the scope.
In the exemplary embodiment of
The method 100 comprises a step S104-A of receiving first input data in relation to a request for medical imaging of a patient.
The method 100 further comprises a step S104-B of receiving second input data indicative of an availability of at least one medical scanner within the fleet of medical scanners.
The method 100 further comprises a step S108 of processing the received S104-A first input data and the received S104-B second input data. The processing S108 comprises selecting an available medical scanner out of the fleet. The processing S108 is performed via a generative artificial intelligence (AI, e.g., comprising a neural network comprising a transformer architecture, and/or a large language model, LLM) for creating a workflow of the medical imaging according to the request.
The method 100 further comprises a step S110 of providing a control signal indicative of the workflow for performing the medical imaging via the selected medical scanner.
Optionally, the method 100 comprises a step S102-A of receiving a technical specification of the medical scanner (and/or preferably for any medical scanner within the fleet of medical scanners). Optionally, the technical specification comprises an existence and/or specification of, in particular detachable, equipment (e.g., one or more coils).
Further optionally, the method 100 comprises a step S102-B of receiving generally applicable instructions in relation to medical imaging by the fleet of medical scanners, in particular comprising one or more medical guidelines and/or one or more standard operation procedures (SOPs).
Still further optionally, the method 100 comprises a step S106 of pre-processing. The pre-processing S106 may comprise pre-processing the received S104-A first input data in relation to the request for medical imaging of the patient. Alternatively or in addition, the pre-processing S106 may comprise pre-processing the received S104-B second input data indicative of the availability of at least one medical scanner within the fleet. Further alternatively or in addition, the pre-processing S106 may comprise pre-processing the received S102-A technical specification of the medical scanner. Still further alternatively or in addition, the pre-processing S106 may comprise pre-processing the received S102-B generally applicable instructions.
The method 100 may further comprise a step S112 of acquiring, by the selected medical scanner, medical imaging data based on the provided S110 control signal.
Alternatively or in addition, the method 100 may comprise a step S114 of post-processing the acquired S112 medical imaging data.
The method 100 may comprise a step S109 of displaying the created workflow using a user interface (UI), in particular a graphical user interface (GUI). Alternatively or in addition, the method 100 may comprise a step S115 of displaying the acquired S112 and/or post-processed S114 medical image data, e.g., using the same or a further UI (in particular the same or a further GUI).
The computing device 200 comprises a first input data receiving interface 204-A configured for receiving first input data in relation to a request for medical imaging of a patient.
The computing device 200 further comprises a second input data receiving interface 204-B configured for receiving second input data indicative of an availability of at least one medical scanner within the fleet of medical scanners.
The computing device 200 further comprises a processing unit 208 configured for processing the received first input data and the received second input data. The processing comprises selecting an available medical scanner out of the fleet. The processing is performed via a generative AI (e.g., by a NN, in particular comprising a transformer architecture, and/or a LLM) for creating a workflow of the medical imaging according to the request.
The computing device 200 further comprises a control signal providing interface 210 configured for providing a control signal indicative of the workflow for performing the medical imaging via the selected medical scanner.
Optionally, the computing device 200 comprises a technical specification receiving interface 202-A configured for receiving a technical specification of the medical scanner, preferably for any medical scanner within the fleet of medical scanners. Optionally, the technical specification comprises an existence and/or specification of, in particular detachable, equipment (e.g., one or more coils).
Further optionally, the computing device 200 comprises a generally applicable instructions receiving interface 202-B configured for receiving generally applicable instructions in relation to medical imaging by the fleet of medical scanners, in particular comprising one or more medical guidelines, and/or one or more SOPs.
Still further optionally, the computing device 200 may comprise a pre-processing unit 208. The pre-processing unit 208 may be configured for pre-processing the received first input data in relation to the request for medical imaging of the patient. Alternatively or in addition, the pre-processing unit 208 may be configured for pre-processing the received second input data indicative of the availability of at least one medical scanner within the fleet. Further alternatively or in addition, the pre-processing unit 208 may be configured for pre-processing the received technical specification of the medical scanner. Still further alternatively or in addition, the pre-processing unit 208 may be configured for pre-processing the received generally applicable instructions.
The computing device 200 may comprise a medical imaging data acquisition interface 212 configured for receiving medical imaging data, which are acquired by the selected medical scanner based on the provided control signal.
The computing device 200 may further comprise a post-processing unit 214 configured for post-processing the acquired medical imaging data.
The computing device 200 may still further comprise a workflow display interface 209 configured for providing the created workflow for display using a UI, in particular a GUI. Alternatively or in addition, the computing device 200 may comprise a medical imaging data display interface 215 configured for providing the acquired and/or post-processed medical image data for display, e.g., on the same or a further UI, in particular the same or a further GUI.
The first input data receiving interface 204-A, the second input data receiving interface 204-B, the optional technical specification receiving interface 202-A, the optional generally applicable instructions receiving interface 202-B, and/or the optional medical imaging data acquisition interface 212 may be comprised in an input interface 216.
The control signal providing interface 210, the optional workflow display interface 209, and/or the optional medical imaging data display interface 215 may be comprised in an output interface 218.
Alternatively or in addition, any one of the interfaces 204-A; 204-B; 202-A; 202-B; 212, and/or 210; 209; 215 may be comprised in an input-output interface 220.
The processing unit 208, the optional pre-processing unit 206, and/or the optional post-processing unit 214 may be embodied by a processor 222.
Any one of the processing unit 208, the optional pre-processing unit 206, the optional post-processing unit 214 and/or the processor 222 may be embodied by a central processing unit (CPU), and/or a graphics processing unit (GPU).
The computing device 200 may further comprise a memory 224.
The computing device 200 may be configured for performing the method 100. E.g., the interfaces 204-A; 204-B, processing unit 208 and interface 210 may be configured to perform the steps S104-A; S104-B; S108 and S110, respectively.
A system may comprise the computing device 200 and a fleet of medical scanners. Each medical scanner within the fleet may comprise a control signal receiving interface configured for receiving a control signal from the control signal providing interface 210 of the computing device 200.
The one or more technical specifications and one or more availability indications received by the technical specification receiving interface 202-A and second input data receiving interface 204-B, respectively, of the computing device 200 may be associated with one or more (in particular each) medical scanner within the fleet.
The system may be configured to perform the method 100.
By the inventive technique (e.g., comprising the method 100, and/or the computing device 200), it is possible to configure and manage (also: control) a fleet of medical scanners (and/or one or more radiology systems). The medical scanners may comprise, e.g., MR devices, CT devices, and/or XR devices.
An AI/AV software (e.g., syngo.via, via, and/or AI Rad Companion, AIRC) and/or reading & reporting software (e.g., Carbon and/or Plaza) may be used in the context (and/or in extensions) of the inventive technique.
The concrete clinical workflows for each patient and imaging procedure can (in particular according to the inventive technique) be optimized end-to-end from scheduling over examination to the reading and reporting processes.
The (e.g., radiology operation) system can (in particular due to the inventive technique) automatically adapt to each of the planned (and/or current and/or future) patient procedure cases, is highly flexible to change in input information and can deal with incidents like patient no-shows or failures of one of more medical scanners (also denoted as device failures).
In
At reference sign 304, a fleet of medical scanners is schematically depicted. At reference sign 306, multiple devices (e.g., workstations) for performing AV functionality are exemplarily shown (e.g., for performing any one of the steps S109; S115). At reference sign 308, multiple instances of devices with AI functionality are schematically depicted (e.g., for performing the post-processing step S114).
Any one of the devices 304; 306; 308 may be comprised in the fleet of radiology devices.
As schematically indicated at reference signs 310 and 312, the acquired medical imaging data may be provided to any one of the devices 306 for performing AV functionality and/or the devices 308 with AI functionality.
As schematically indicated at reference sign 314, the devices 306 and 308 may work combinedly (and/or may be combined in joint device) for performing AI/AV functionalities.
Any one of the (e.g., input and/or acquired medical imaging) data forwarding (e.g., as indicated by the arrows in
The inference (e.g., in the step S108 of the method 100) may comprise choosing the right scanner, configuring the right scan parameters, standardizing reading output, triggering the right AI/AV algorithms fitting to the clinical condition of the patient, displaying the data reproducibly in the appropriate views and/or hangings, offering the right tools, and preparing and/or transmitting a report to the referrers. “right” herein may be synonymously used to “suitable”, “appropriate”, and/or in particular “optimal for the request”.
Natural language processing (NLP) techniques may automatically analyze medical documents, e.g., reports, including radiology reports, and/or referral letters. One technical realization is Large Language Models (LLMs). Such LLMs have been shown to be effective at tasks such as text generation and language translation. LLMs have also been applied to the analysis of medical texts, including the extraction of information from Electronic Medical Records (EMR), in particular Miotto et al., 2016 [9], which is incorporated herein by reference.
The inventive technique (e.g., comprising the method 100 and/or computing device 200) makes use of algorithms, AI, and/or (e.g., preferably) LLMs to analyze multiple information sources. Any combination of information sources makes sense, as exemplified at reference signs 302-1; 302-2; 302-3; 302-4; 302-5 in
The prior patient information 302-1 may, e.g., comprise referral orders, Electronic Health Record (HER) patient information, images, radiology reports, and/or lab reports.
The prior information 302-1, especially reports and/or referrals, may be analyzed with (in particular generative) AI 308 and/or LLMs. The analysis of the prior information 302-1 may provide the request for medical imaging and/or an indication for examination.
The analysis of the prior information 302-1 may provide equipment, tools, scan parameters and/or measurements used in prior exams, steps before, and/or for similar patients. Similar patients may related to a similar reason for the medical imaging (e.g., a similar expected diagnosis, and/or a similar accident).
Alternatively or in addition, the prior information 302-1 may provide views used and/or created in prior exams, steps before, and/or for similar patients.
Alternatively or in addition, the prior information 302-1 may provide quantitative data obtained manually, semi-automatically, and/or automatically (e.g., by using software tools like, e.g. AIRC, via, and/or syngo.via on current and prior data) that are relevant to the current examination.
Alternatively or in addition, the prior information 302-1 may provide a view configuration used in prior exams, steps before, and/or for similar patients.
Alternatively or in addition, the prior information 302-1 may provide prior radiology reports for the same patient, to whom the request is related, which relate to the reason for the medical imaging exam and/or for similar patients.
Alternatively or in addition, the prior information 302-1 may provide prior medical examinations, e.g., blood values, reasons for exams and surgeries, in particular in relation to the patient, to whom the request refers.
The prior information 302-1 may be outputted in a structured way. The following example shows how to prompt a LLM (e.g., GPT4-32k) to create a JavaScript Object Notation (JSON) structure of relevant findings out of an unstructured report:
The output of the above example prompt in relation to the report reads as follows:
The capabilities of the medical scanners and/or equipment (and/or more generally assets) involved play a key role in defining the optimal workflow.
The capabilities (e.g., according to the technical specification received in the step S102-A) of an MRT scanner (also denoted as Magnetic Resonance Imaging, MRI, device) may comprise hardware and software capabilities, e.g., a magnet field strength, a gradient strength, one or more receiver channels, one or more available radio frequency (rf)-coils, one or more available MRI sequences and/or protocols, and/or available software licenses.
The capabilities (e.g., according to the technical specification received in the step S102-A) of a CT scanner (also denoted as CT device) may comprise hardware and software capabilities, e.g., (in particular a number of) slices, a detector width, dual energy capabilities, a number of coils, photon counting capabilities, physiological triggering, available CT software and/or protocols.
The capabilities (e.g., for the post-processing step S114) of the AI/AV (e.g., on the medical scanner, on-premise, and/or on cloud) may comprise hardware and software capabilities, e.g., an availability of e.g., lung or colon CAD, a brain hemorrhage detection, a coronary artery assessment, and/or number of available seats and/or users.
The capabilities (e.g., for the post-processing step S114, and/or for any one of the displaying steps S109 and S115) for reading and/or reporting may comprise hardware and software capabilities, e.g., a monitor configuration available, one or more AI/AV tools, and/or rendering capabilities.
Alternatively, information on the assets may comprise an (e.g., instantaneous, and/or time-limited) availability.
The information on the capabilities, and/or on the availability, may exist, and/or may be provided, in a digitally structured way. The assets (e.g., the medical scanners within the fleet, equipment, AI/AV, and/or reading and reporting tools) involved may require an Application Procedure Interface (API) to bring their information to the inference level in a structured way. The information may be standardized and put into a regularly updated database. Real-time status information may be included in a preferable implementation, e.g., describing current and future availability of the asset (e.g., a medical scanner within the fleet, equipment, AI/AV, and/or a reading and reporting tool) for use.
For two MRT scanners, the standardized information may exemplarily read as follows:
The structured information may still require (and/or need) an ontology (e.g., for a radiology device and/or medical device), in particular comprising technical specifications (e.g., of medical scanners across multiple medical imaging modalities), to make capabilities comparable.
Referral orders must be (and/or are usually) entered into the fleet (and/or the radiology entity's) schedule. Commonly, Electronic Health Record (EHR) scheduling systems (e.g., mostly in the United States of America), legacy Radiology Information Systems (RIS, e.g., both in the United States of America and Europe), and/or (e.g., B2C) calendar (and/or scheduling) tools, like Outlook, may be used to manage the scheduling slots for patients and/or (e.g., often also) personnel.
On-site personnel usually interacts with patients directly (e.g., in outpatient settings) and/or with clinical departments (e.g., in inpatient settings) to schedule an appointment for the medical imaging (and/or radiology procedure). It may conventionally easily happen that multiple (e.g., five) calendars need to be coordinated to realize a procedure.
The inventive technique (e.g., comprising the method 100, the computing device 200, and/or the, in particular radiology operation, system) in a preferred embodiment uses patient scheduling information to plan and adjust individual procedure workflows. The scheduling information may be a precise time slot or a general availability.
In another preferred embodiment, the inventive technique (e.g., comprising the method 100, the computing device 200, and/or the, in particular radiology operation, system) comprises a back channel from the action layer to the scheduling system, so that incidents, like a delayed scan and/or breakdown of a medical scanner (and/or system), may be relayed back to the scheduling system and the patients (e.g., as indicated at reference sign S109 in
Conventional fleet management solutions usually provide an analysis of scanner productivity based on the DICOM data created. Some solutions (like teamplay insights) also use log data. However, using the DICOM data, and/or the use log data, is a costly and tedious activity as the log syntax is mostly non-standardized and needs manually created filters to import such data into (in particular site-specific) performance metrics.
Using algorithms, in particular a generative AI (preferably LLMs), it can be derived which scan parameters are used where, how real slot times are, and/or what image contrasts are created based on which scan parameters. Any one of these types of information may be standardized and put into a regularly updated database.
Preferences by the operator of the fleet of medical scanners may be expressed by behavioral patterns (e.g., by techs and/or radiologists), SOPs, and/or clinical (and/or legal) guidelines (briefly: guidelines). The clinical (and/or legal) guidelines are conventionally expressed as written documents. While actual behavior is usually logged in the (in particular site-specific) performance metrics, the SOP and guideline-based information is usually not accessible for automated workflow optimization. Manual configuration of radiology devices is usually not performed. E.g., conventionally, DICOM-routers are not (in particular manually) configured for optimizing the workflow. Alternatively or in addition, settings for performing (and/or when to perform) Lung CAD are conventionally not configured. Further alternatively or in addition, conventionally, the configuration for (in particular image) reconstruction is not (in particular manually) optimized.
Using the generative AI (e.g., one or more LLMs), SOPs and/or guidelines can be interpreted out of human language and converted (and/or standardized, and/or normalized) into machine interpretable structures. Preferably, the conversion (also: standardization and/or normalization) is combined with a classification algorithm (e.g., an image foundational model) that can classify images. Such information (e.g., comprising the classification, standardized SOPs, and/or standardized guidelines) may, e.g., comprise rules on how to view a certain case in a PACS. E.g., a SOP may comprise the definition of a so-called “hanging” or “hanging protocol”, as exemplified in
In
The (in particular previous hanging, and/or parts thereof) may be converted into, e.g.:
By providing an identical hanging, the visual inspection and/or diagnosis by a medical practitioner (in particular the referrer) is improved.
The selection of views and/or of the hanging may, e.g., be performed by one of the computers 306 in
On the inference level, the information level data may be combined to create and constantly update the optimal workflow for each patient procedure.
In an embodiment, an MRT imaging procedure is configured and planned based on the referral order (in particular comprised in the first input data received in the step 104-A) and scheduling information (in particular comprised in the second input data received in the step 104-B) as well as additional patient information (in particular also comprised in the first input data received in the step 104-A), like age and MRT scan counter-indications, and the radiology's SOPs (in particular comprised in the generally application instruction in the step S102-B).
In a further embodiment, a CT imaging procedure is configured and planned on the best available (or good enough) scanner for the imaging task. A Photon Counting CT might be preferred due to the advantages in resolution, a dual-source CT might be preferred to perform a cardiac examination, and/or a wide bore scanner may be preferred for an obese patient.
In another embodiment comprising the example for reading hangings according to
An exemplary workflow (and/or workflow plan) may read as follows in human language:
Based on the patient's history and symptoms, it is advised to perform CT scans on the following areas:
Auto-processings and image-based AI features to be prepared for the radiologist:
Converted (in particular automatically) into a machine readable format, the exemplary workflow (and/or workflow plan) may read as follows:
On the action level (e.g., as indicated at reference signs S110; S112 in
All actions following the plan of the inference level may fail occasionally. E.g., radiology employee might be pulled into an emergency case, a patient might not show up, and/or a medical scanner (and/or device) might malfunction.
All failures and/or incidents may change the information level (in particular the availability 302-5 of one of the medical scanners 304 indicated in the second input data received at step S104-B in
A parametrization for a (in particular concrete) medical scanner is exemplarily shown in
As medical scanners conventionally cannot process (and/or do not take in) scan parameters in parametrizations for documentation and/or human use (e.g., as displayed in
In a preferred embodiment, the (in particular radiology operation) system may display (also: show, e.g., according to the method step S109) both the planned and actual workflows for each patient and an aggregated overview of the whole radiology asset (e.g., comprising medical scanners and/or, in particular detachable, equipment) usage. The radiologist and tech can access both current and past individual workflows and comprehensive performance overviews.
In another preferred embodiment, the output and human interaction (e.g., using a UI, in particular GUI) of the (e.g., radiology operation) system are carried out (and/or done) via mobile devices, e.g., mobile phones, tablets, smart watches, smart glasses, and/or smart contact lenses.
In a further preferred embodiment, the output and human interaction of the (e.g., radiology operation) system are carried out (and/or done) via optical and/or touch units at the medical scanner (e.g., using a UI, in particular GUI, at the medical scanner).
The backchannel signal may be processed during an inference phase.
The backchannel signal may comprise the second input data, indicative of an—in this case amended—availability of the radiology device (e.g., medical scanner 304). The backchannel signal may be received on the scanner availability receiving interface 204-B.
The backchannel signal may comprise the technical specification of the medical scanner 304. The backchannel signal may be received on the technical specification interface 202-A.
The backchannel signal is transmitted from the medical scanner 304 to the computing device 200 and/or the control signal is transmitted (as a result of processing) from the computing device 200 to the medical scanner 304 for controlling the latter.
Any one of the embodiments may be combinable with each other.
The inventive technique (e.g., comprising the method 100, and/or the computing device 200) may be instantiated in a cloud infrastructure. Thereby, a widely distributed fleet of assets (e.g., medical scanners and/or, in particular detachable, equipment) can be addressed, managed, scheduled, and/or controlled.
The inventive technique (e.g., comprising the method 100, the computing device 200, and/or the, in particular radiology operation, system) combines the multiple information sources (e.g., comprising the first input data in relation to the request for medical imaging, the second input data indicative of the availability of one or more medical scanners within the fleet, the technical specification of any medical scanner within the fleet, and/or the generally applicable instructions, e.g., comprising guidelines and/or SOPs) and creates the optimal workflow (also: workflow plan). The inventive technique furthermore allows execution of dynamic, patient adjusted and/or personalized workflows and parametrizations using the available assets (e.g., medical scanners within the fleet and/or, in particular detachable, equipment). The inventive technique avoids the need for complicated individual and local configurations of the assets. The configuration itself can be based on, e.g., human readable documents, SOPs, and/or guidelines, and can adopt to current practice and/or behavior out of the existing prior data. The behavior may comprise a choice of scan parameters and/or settings for the requested medical imaging.
The inventive technique may be directly visible and/or detectable on how the implementation is configured and instructed, e.g., including the data connectivity.
Wherever not already described explicitly, individual embodiments, or their individual aspects and features, described in relation to the drawings can be combined or exchanged with one another without limiting or widening the scope of the described invention, whenever such a combination or exchange is meaningful and in the sense of this invention. Advantages which are described with respect to a particular embodiment of present invention or with respect to a particular figure are, wherever applicable, also advantages of other embodiments of the present invention.
It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, components, regions, layers, and/or sections, these elements, components, regions, layers, and/or sections, should not be limited by these terms. These terms are only used to distinguish one element from another. For example, a first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of example embodiments. As used herein, the term “and/or,” includes any and all combinations of one or more of the associated listed items. The phrase “at least one of” has the same meaning as “and/or”.
Spatially relative terms, such as “beneath,” “below,” “lower,” “under,” “above,” “upper,” and the like, may be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as “below,” “beneath,” or “under,” other elements or features would then be oriented “above” the other elements or features. Thus, the example terms “below” and “under” may encompass both an orientation of above and below. The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly. In addition, when an element is referred to as being “between” two elements, the element may be the only element between the two elements, or one or more other intervening elements may be present.
Spatial and functional relationships between elements (for example, between modules) are described using various terms, including “on,” “connected,” “engaged,” “interfaced,” and “coupled.” Unless explicitly described as being “direct,” when a relationship between first and second elements is described in the disclosure, that relationship encompasses a direct relationship where no other intervening elements are present between the first and second elements, and also an indirect relationship where one or more intervening elements are present (either spatially or functionally) between the first and second elements. In contrast, when an element is referred to as being “directly” on, connected, engaged, interfaced, or coupled to another element, there are no intervening elements present. Other words used to describe the relationship between elements should be interpreted in a like fashion (e.g., “between,” versus “directly between,” “adjacent,” versus “directly adjacent,” etc.).
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms “a,” “an,” and “the,” are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the terms “and/or” and “at least one of” include any and all combinations of one or more of the associated listed items. It will be further understood that the terms “comprises,” “comprising,” “includes,” and/or “including,” when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. Expressions such as “at least one of,” when preceding a list of elements, modify the entire list of elements and do not modify the individual elements of the list. Also, the term “example” is intended to refer to an example or illustration.
It should also be noted that in some alternative implementations, the functions/acts noted may occur out of the order noted in the figures. For example, two figures shown in succession may in fact be executed substantially concurrently or may sometimes be executed in the reverse order, depending upon the functionality/acts involved.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which example embodiments belong. It will be further understood that terms, e.g., those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
It is noted that some example embodiments may be described with reference to acts and symbolic representations of operations (e.g., in the form of flow charts, flow diagrams, data flow diagrams, structure diagrams, block diagrams, etc.) that may be implemented in conjunction with units and/or devices discussed above. Although discussed in a particular manner, a function or operation specified in a specific block may be performed differently from the flow specified in a flowchart, flow diagram, etc. For example, functions or operations illustrated as being performed serially in two consecutive blocks may actually be performed simultaneously, or in some cases be performed in reverse order. Although the flowcharts describe the operations as sequential processes, many of the operations may be performed in parallel, concurrently or simultaneously. In addition, the order of operations may be re-arranged. The processes may be terminated when their operations are completed, but may also have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, subprograms, etc.
Specific structural and functional details disclosed herein are merely representative for purposes of describing example embodiments. The present invention may, however, be embodied in many alternate forms and should not be construed as limited to only the embodiments set forth herein.
In addition, or alternative, to that discussed above, units and/or devices according to one or more example embodiments may be implemented using hardware, software, and/or a combination thereof. For example, hardware devices may be implemented using processing circuitry such as, but not limited to, a processor, Central Processing Unit (CPU), a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a System-on-Chip (SoC), a programmable logic unit, a microprocessor, or any other device capable of responding to and executing instructions in a defined manner. Portions of the example embodiments and corresponding detailed description may be presented in terms of software, or algorithms and symbolic representations of operation on data bits within a computer memory. These descriptions and representations are the ones by which those of ordinary skill in the art effectively convey the substance of their work to others of ordinary skill in the art. An algorithm, as the term is used here, and as it is used generally, is conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of optical, electrical, or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.
It should be borne in mind that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise, or as is apparent from the discussion, terms such as “processing” or “computing” or “calculating” or “determining” of “displaying” or the like, refer to the action and processes of a computer system, or similar electronic computing device/hardware, that manipulates and transforms data represented as physical, electronic quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.
In this application, including the definitions below, the term ‘module’ or the term ‘controller’ may be replaced with the term ‘circuit.’ The term ‘module’ may refer to, be part of, or include processor hardware (shared, dedicated, or group) that executes code and memory hardware (shared, dedicated, or group) that stores code executed by the processor hardware.
The module may include one or more interface circuits. In some examples, the interface circuits may include wired or wireless interfaces that are connected to a local area network (LAN), the Internet, a wide area network (WAN), or combinations thereof. The functionality of any given module of the present disclosure may be distributed among multiple modules that are connected via interface circuits. For example, multiple modules may allow load balancing. In a further example, a server (also known as remote, or cloud) module may accomplish some functionality on behalf of a client module.
Software may include a computer program, program code, instructions, or some combination thereof, for independently or collectively instructing or configuring a hardware device to operate as desired. The computer program and/or program code may include program or computer-readable instructions, software components, software modules, data files, data structures, and/or the like, capable of being implemented by one or more hardware devices, such as one or more of the hardware devices mentioned above. Examples of program code include both machine code produced by a compiler and higher level program code that is executed using an interpreter.
For example, when a hardware device is a computer processing device (e.g., a processor, Central Processing Unit (CPU), a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a microprocessor, etc.), the computer processing device may be configured to carry out program code by performing arithmetical, logical, and input/output operations, according to the program code. Once the program code is loaded into a computer processing device, the computer processing device may be programmed to perform the program code, thereby transforming the computer processing device into a special purpose computer processing device. In a more specific example, when the program code is loaded into a processor, the processor becomes programmed to perform the program code and operations corresponding thereto, thereby transforming the processor into a special purpose processor.
Software and/or data may be embodied permanently or temporarily in any type of machine, component, physical or virtual equipment, or computer storage medium or device, capable of providing instructions or data to, or being interpreted by, a hardware device. The software also may be distributed over network coupled computer systems so that the software is stored and executed in a distributed fashion. In particular, for example, software and data may be stored by one or more computer readable recording mediums, including the tangible or non-transitory computer-readable storage media discussed herein.
Even further, any of the disclosed methods may be embodied in the form of a program or software. The program or software may be stored on a non-transitory computer readable medium and is adapted to perform any one of the aforementioned methods when run on a computer device (a device including a processor). Thus, the non-transitory, tangible computer readable medium, is adapted to store information and is adapted to interact with a data processing facility or computer device to execute the program of any of the above mentioned embodiments and/or to perform the method of any of the above mentioned embodiments.
Example embodiments may be described with reference to acts and symbolic representations of operations (e.g., in the form of flow charts, flow diagrams, data flow diagrams, structure diagrams, block diagrams, etc.) that may be implemented in conjunction with units and/or devices discussed in more detail below. Although discussed in a particular manner, a function or operation specified in a specific block may be performed differently from the flow specified in a flowchart, flow diagram, etc. For example, functions or operations illustrated as being performed serially in two consecutive blocks may actually be performed simultaneously, or in some cases be performed in reverse order.
According to one or more example embodiments, computer processing devices may be described as including various functional units that perform various operations and/or functions to increase the clarity of the description. However, computer processing devices are not intended to be limited to these functional units. For example, in one or more example embodiments, the various operations and/or functions of the functional units may be performed by other ones of the functional units. Further, the computer processing devices may perform the operations and/or functions of the various functional units without sub-dividing the operations and/or functions of the computer processing units into these various functional units.
Units and/or devices according to one or more example embodiments may also include one or more storage devices. The one or more storage devices may be tangible or non-transitory computer-readable storage media, such as random access memory (RAM), read only memory (ROM), a permanent mass storage device (such as a disk drive), solid state (e.g., NAND flash) device, and/or any other like data storage mechanism capable of storing and recording data. The one or more storage devices may be configured to store computer programs, program code, instructions, or some combination thereof, for one or more operating systems and/or for implementing the example embodiments described herein. The computer programs, program code, instructions, or some combination thereof, may also be loaded from a separate computer readable storage medium into the one or more storage devices and/or one or more computer processing devices using a drive mechanism. Such separate computer readable storage medium may include a Universal Serial Bus (USB) flash drive, a memory stick, a Blu-ray/DVD/CD-ROM drive, a memory card, and/or other like computer readable storage media. The computer programs, program code, instructions, or some combination thereof, may be loaded into the one or more storage devices and/or the one or more computer processing devices from a remote data storage device via a network interface, rather than via a local computer readable storage medium. Additionally, the computer programs, program code, instructions, or some combination thereof, may be loaded into the one or more storage devices and/or the one or more processors from a remote computing system that is configured to transfer and/or distribute the computer programs, program code, instructions, or some combination thereof, over a network. The remote computing system may transfer and/or distribute the computer programs, program code, instructions, or some combination thereof, via a wired interface, an air interface, and/or any other like medium.
The one or more hardware devices, the one or more storage devices, and/or the computer programs, program code, instructions, or some combination thereof, may be specially designed and constructed for the purposes of the example embodiments, or they may be known devices that are altered and/or modified for the purposes of example embodiments.
A hardware device, such as a computer processing device, may run an operating system (OS) and one or more software applications that run on the OS. The computer processing device also may access, store, manipulate, process, and create data in response to execution of the software. For simplicity, one or more example embodiments may be exemplified as a computer processing device or processor; however, one skilled in the art will appreciate that a hardware device may include multiple processing elements or processors and multiple types of processing elements or processors. For example, a hardware device may include multiple processors or a processor and a controller. In addition, other processing configurations are possible, such as parallel processors.
The computer programs include processor-executable instructions that are stored on at least one non-transitory computer-readable medium (memory). The computer programs may also include or rely on stored data. The computer programs may encompass a basic input/output system (BIOS) that interacts with hardware of the special purpose computer, device drivers that interact with particular devices of the special purpose computer, one or more operating systems, user applications, background services, background applications, etc. As such, the one or more processors may be configured to execute the processor executable instructions.
The computer programs may include: (i) descriptive text to be parsed, such as HTML (hypertext markup language) or XML (extensible markup language), (ii) assembly code, (iii) object code generated from source code by a compiler, (iv) source code for execution by an interpreter, (v) source code for compilation and execution by a just-in-time compiler, etc. As examples only, source code may be written using syntax from languages including C, C++, C#, Objective-C, Haskell, Go, SQL, R, Lisp, Java®, Fortran, Perl, Pascal, Curl, OCaml, Javascript®, HTML5, Ada, ASP (active server pages), PHP, Scala, Eiffel, Smalltalk, Erlang, Ruby, Flash®, Visual Basic®, Lua, and Python®.
Further, at least one example embodiment relates to the non-transitory computer-readable storage medium including electronically readable control information (processor executable instructions) stored thereon, configured in such that when the storage medium is used in a controller of a device, at least one embodiment of the method may be carried out.
The computer readable medium or storage medium may be a built-in medium installed inside a computer device main body or a removable medium arranged so that it can be separated from the computer device main body. The term computer-readable medium, as used herein, does not encompass transitory electrical or electromagnetic signals propagating through a medium (such as on a carrier wave); the term computer-readable medium is therefore considered tangible and non-transitory. Non-limiting examples of the non-transitory computer-readable medium include, but are not limited to, rewriteable non-volatile memory devices (including, for example flash memory devices, erasable programmable read-only memory devices, or a mask read-only memory devices); volatile memory devices (including, for example static random access memory devices or a dynamic random access memory devices); magnetic storage media (including, for example an analog or digital magnetic tape or a hard disk drive); and optical storage media (including, for example a CD, a DVD, or a Blu-ray Disc). Examples of the media with a built-in rewriteable non-volatile memory, include but are not limited to memory cards; and media with a built-in ROM, including but not limited to ROM cassettes; etc. Furthermore, various information regarding stored images, for example, property information, may be stored in any other form, or it may be provided in other ways.
The term code, as used above, may include software, firmware, and/or microcode, and may refer to programs, routines, functions, classes, data structures, and/or objects. Shared processor hardware encompasses a single microprocessor that executes some or all code from multiple modules. Group processor hardware encompasses a microprocessor that, in combination with additional microprocessors, executes some or all code from one or more modules. References to multiple microprocessors encompass multiple microprocessors on discrete dies, multiple microprocessors on a single die, multiple cores of a single microprocessor, multiple threads of a single microprocessor, or a combination of the above.
Shared memory hardware encompasses a single memory device that stores some or all code from multiple modules. Group memory hardware encompasses a memory device that, in combination with other memory devices, stores some or all code from one or more modules.
The term memory hardware is a subset of the term computer-readable medium. The term computer-readable medium, as used herein, does not encompass transitory electrical or electromagnetic signals propagating through a medium (such as on a carrier wave); the term computer-readable medium is therefore considered tangible and non-transitory. Non-limiting examples of the non-transitory computer-readable medium include, but are not limited to, rewriteable non-volatile memory devices (including, for example flash memory devices, erasable programmable read-only memory devices, or a mask read-only memory devices); volatile memory devices (including, for example static random access memory devices or a dynamic random access memory devices); magnetic storage media (including, for example an analog or digital magnetic tape or a hard disk drive); and optical storage media (including, for example a CD, a DVD, or a Blu-ray Disc). Examples of the media with a built-in rewriteable non-volatile memory, include but are not limited to memory cards; and media with a built-in ROM, including but not limited to ROM cassettes; etc. Furthermore, various information regarding stored images, for example, property information, may be stored in any other form, or it may be provided in other ways.
The apparatuses and methods described in this application may be partially or fully implemented by a special purpose computer created by configuring a general purpose computer to execute one or more particular functions embodied in computer programs. The functional blocks and flowchart elements described above serve as software specifications, which can be translated into the computer programs by the routine work of a skilled technician or programmer.
Although described with reference to specific examples and drawings, modifications, additions and substitutions of example embodiments may be variously made according to the description by those of ordinary skill in the art. For example, the described techniques may be performed in an order different with that of the methods described, and/or components such as the described system, architecture, devices, circuit, and the like, may be connected or combined to be different from the above-described methods, or results may be appropriately achieved by other components or equivalents.
| Number | Date | Country | Kind |
|---|---|---|---|
| 23210160.0 | Nov 2023 | EP | regional |
