Patent Application
20220248973
The invention relates to scanning imaging systems, in particular to a medical analysis system for determining electrical properties (EP) of a target volume in an imaged subject.
Electric Properties Tomography (EPT) is a technique to measure electric properties of tissue non-invasively and quantitatively in-vivo by post-processing of the magnitude and phase of B1 field distributions. The relation between EPs and complex B1 values is given by the so-called Helmholtz-equation. Thus, taking the complex valued B1 map from MRI as input, the Helmholtz equation provides the EPs. However, conventional EPTs suffer from the transceive phase problem, issues at tissue boundaries and noise amplification.
Various embodiments provide for a medical analysis system for determining electrical properties (EP) of a target volume in an imaged subject, method, and computer program product, as described by the subject matter of the independent claims. Advantageous embodiments are described in the dependent claims.
Embodiments of the invention may provide for a means of using domain-adversarial training for attaining a robust deep learning of EPT to artifacts arising from in-vivo image acquisition. This may be achieved by performing an adversarial training. Along with a reconstruction network (generator) an additional network (discriminator) is trained to identify high level deviations between generator outputs and realistic tissue EPs (e.g. electromagnetic models incorporating literature EP values). For example, after pre-training the generator on simulations, in-vivo B1 data without EP labels but with in-vivo artifacts is utilized for training the generator to create outputs which the discriminator cannot distinguish from realistic output.
In one aspect, the invention relates to a medical analysis system for determining electrical properties (EP) of a target volume in a subject. For example, magnetic resonance electrical property tomography (MR-EPT) may be used for estimating the conductivity and permittivity distribution of the target volume. Imaging in accordance with MR-EPT may enable to obtain a phase image and/or B1 amplitude image of the RF field produced using specific pulse sequences. For example, the subject may be imaged by a magnetic resonance imaging (MRI) system, resulting in MRI data. The MRI data may be used to reconstruct tissue EPs by solving an electromagnetic inverse problem relating MR measurements of the transmit radio frequency (RF) field to the EPs. The medical analysis system comprises at least one processor; and at least one memory storing machine executable instructions. The processor is configured for controlling the medical analysis system, wherein execution of the machine executable instructions causes the processor to: perform a first training of a deep neural network, DNN, using a first training dataset, the first training dataset comprising training B1 field maps and corresponding first EP maps, the first training resulting in a pre-trained DNN configured for generating EP maps from B1 field maps; perform a second training of the pre-trained DNN using conditional generative adversarial networks, GAN, and a second training dataset, wherein the pre-trained DNN is a generator of the conditional GAN, the second training dataset comprising measured B1 maps and second EP maps, the second training resulting in a trained DNN; receive an input B1 field map of the target volume and generating an EP map of the input B1 field map using the trained DNN.
Numerous B1 mapping methods exist for experimentally determining B1 field distributions. These methods may be used for measuring magnitude and phase distributions. For example, two MRI scans may be used to acquire or generate a complex-valued B1 field map e.g. one scan for acquiring the B1 magnitude distribution and one scan for acquiring the B1 phase distribution. The B1 field maps may be B1 transmit field maps.
When the information on B1 field is determined or presented in the form of a 2-D or 3-D spatial distribution, it may be called a B1 field map. The B1 field map may for example be determined using MRI data acquired by scanning the target volume of the subject. A B1 field map may be used to derive from the B1 field map an EP map. The derived EP map therefore corresponds to that B1 field map. Deriving the EP map comprises performing an EPT reconstruction using the B1 field map e.g. the EP map may be computed by taking Laplacian of the B1 field map.
By performing a two-stage training, the present subject matter may enable to obtain accurate tissue electric properties (conductivity and permittivity). This may particularly be advantageous as the electric properties may be critical for correct electromagnetic simulations and subsequent specific absorption rate (SAR) assessment such as for safety assessment of MRI. The present subject matter may enable noninvasive in-vivo measurements.
The present subject matter may be advantageous compared to existing methods, as they show a large variation between the reported electrical properties. These variations can be explained by the use of tissues of various species and variations in measuring conditions (e.g. tissue temperature, in vivo, in vitro and ex vivo). The present method may be used to verify the validity of the current maintained in vivo electric property values. In particular, utilization of deep learning for EPT may solve essential problems of the conventional methods for EPT such as the transceive phase problem, issues at tissue boundaries and noise amplification.
The present subject matter may be applied for cases where electric properties are of diagnostic value, or where electric properties are needed for RF safety management, or where electric properties are needed for therapy planning like RF ablation/hyperthermia.
According to one embodiment, each of the training and measured B1 field maps comprises a respective B1 field phase map and/or B1 amplitude map, wherein the input B1 field map comprises a B1 phase map and/or B1 amplitude map.
In a first example, each of the training and measured B1 field maps comprises a respective B1 field phase map, wherein the input B1 field map comprises a B1 phase map. In other terms, the first and second training are performed using B1 phase maps in order to generate a trained DNN that can estimate or predict a conductivity map in response to receiving as input a B1 phase map. The DNN may for example comprise an input layer configured to receive voxel or pixel values of the training B1 maps. Each training B1 map may for example comprise an image whose pixel values may be input to the DNN for performing the first training. The output layer of the DNN may for example be configured to provide output values which are indicative of the conductivity values. The first training may result in the DNN being trained for outputting a prediction or estimation of a conductivity map that corresponds to an input B1 field phase map. This may enable an initialization of the DNN (e.g. initialized weights) before being trained in the GAN framework. The initialization with the first training may improve the accuracy of the second training and thus the predicted EP maps may be more accurate. The input B1 field map may be an input B1 phase map. By receiving the input B1 phase map, the trained DNN may output the conductivity map. This may allow an accurate and precise conductivity map reconstruction e.g. of human brain tissues at clinically available MR field strengths.
In a second example, each of the training and measured B1 field maps comprises a respective B1 amplitude map, wherein the input B1 field map comprises a B1 amplitude map. In other terms, the first and second training are performed using B1 amplitude maps in order to generate a trained DNN that can estimate or predict a permittivity map in response to receiving as input a B1 amplitude map. The DNN may for example comprise an input layer configured to receive voxel or pixel values of the B1 amplitude maps. Each B1 amplitude map may for example comprise an image whose pixel values may be input to the DNN for performing the first training. The output layer of the DNN may for example be configured to provide output values which are indicative of the permittivity values. The first training may result in the DNN being trained for outputting a prediction or estimation of a permittivity map that corresponds to an input B1 amplitude map. This may enable an initialization of the DNN (e.g. initialized weights) before being trained in the GAN framework. The input B1 field map may be an input B1 amplitude map. By receiving the input B1 amplitude map, the trained DNN may output the permittivity map. This may allow an accurate and precise permittivity map reconstruction e.g. of human brain tissues at clinically available MR field strengths.
The first and second examples may enable a phase-only conductivity reconstruction and a magnitude-only permittivity reconstruction. This may particularly be advantageous in the frequency range corresponding to MR field strengths below 3 T, where the wave behavior of B1 is less significant, and the absolute B1 phase depends on the conductivity distribution more than the permittivity while permittivity is more closely related to the magnitude of B1 fields.
In a third example, each of the training and measured B1 field maps comprises a respective B1 field phase map and B1 amplitude map, wherein the input B1 field map comprises a B1 phase map and B1 amplitude map. In other terms, the first and second training are performed using both B1 amplitude and B1 phase maps in order to generate a trained DNN that can estimate or predict a permittivity map and conductivity map in response to receiving as input a B1 amplitude and B1 phase map. The DNN may for example comprise an input layer configured to receive voxel or pixel values of both the B1 amplitude and B1 phase maps. Each B1 amplitude map and B1 phase map may for example comprise an image, wherein corresponding pixel values of the two images may be input to the input layer of the DNN for performing the first training. The output layer of the DNN may for example be configured to provide output values which are indicative of the permittivity and conductivity values. The first training may result in the DNN being trained for outputting a prediction or estimation of a permittivity map and conductivity map that correspond to an input B1 field map. This may enable an initialization of the DNN (e.g. initialized weights) before being trained in the GAN framework. By receiving the input B1 field map, the trained DNN may output the permittivity and conductivity maps. This may allow an accurate and precise permittivity map reconstruction e.g. of human brain tissues at clinically available MR field strengths.
According to one embodiment, the training B1 maps comprise B1 amplitude maps, wherein the pre-trained DNN is configured to generate permittivity maps from B1 amplitude maps. The first training is limited to the B1 amplitude maps while the second training may use both the B1 amplitude and B1 phase maps. This may enable to speed up the process of predicting the EP maps while still providing accurate and precise EP map reconstructions.
According to one embodiment, the training B1 maps comprise B1 phase maps, wherein the pre-trained DNN is configured to generate conductivity maps from B1 phase maps. The first training is limited to the B1 phase maps while the second training may use both the B1 amplitude and B phase maps. This may enable to speed up the process of predicting the EP maps while still providing accurate and precise EP map reconstructions.
According to one embodiment, the training B1 field maps comprise simulated B1 field maps and associated simulated EP maps, the second EP maps comprising simulated EP maps, wherein the second training comprises training the generator using the measured B1 field maps and training a discriminator of the GAN using both the simulated EP maps and EP maps generated by the generator. This may enable a conditional GAN framework in which the generator is conditioned on the measured B1 field maps. The simulated B1 field maps do not comprise artifacts.
For example, the discriminator looks at simulated EP maps (as real EP maps) and generated EP maps separately. The discriminator may distinguish whether an input EP map is real or generated. This may enable that the generator is conditioned with measured B1 field maps. This may enable that, instead of generating EP maps from noise only, measured B1 field maps may be combined with noise and may be used by the generator to generate the EP maps. The training of the generator and discriminator of the GAN may then be performed in an adversarial way as with training the GAN, but the generator is conditioned with the measured B1 field maps. This may enable a better convergence of the training.
Both networks of the discriminator and the generator may be trained in alternating steps. For example, the generator's parameters may be fixed and a single iteration of gradient descent on the discriminator is performed using the real and the generated EP maps. Then the discriminator's parameters may be fixed and the generator may be trained for another iteration.
Straightforward training for a DNN with electromagnetic simulations without considering artifacts from in-vivo image acquisition may result in reconstruction errors when applying networks trained on simulations to in-vivo data. On the other hand, including simulations of in-vivo artifacts in training explicitly may be a cumbersome and lengthy procedure, and may have has problems with authenticity, and includes the risk of missing unidentified artefacts. The present subject matter may overcome such issues by applying a two stages training by first (pre) training the DNN using simulation data without artifacts and second training the pre-trained DNN using measured data that has artifacts.
The training of the discriminator comprises a backpropagation of an error on the determination if the input is a real or fake (generated) EP map. The training of the generator comprises a backpropagation of an error of the classification of an error on the determination if the input is a real or fake EP map. The weights of each of the networks of the discriminator and the generator may be updated based on the errors back propagated.
The simulation data may be obtained by simulating a realistic coil setup and including realistic head models. This may enable to obtain a high number of unique B1 fields. This may overcome the need of a high amount of MR data for training.
According to one embodiment, generating the EP map of the input B1 field map is performed using the trained DNN and the pre-trained DNN, the generating comprising: generating the EP map using each of the trained DNN and the pre-trained DNN; averaging the generated EP maps and providing an uncertainty on the averaged EP map. This may enable an access to uncertainties of the predicted EP maps and may thus provide means for checking the accuracy of the predicted EP maps.
According to one embodiment, the DNN is a U-NET. The discriminator of the GAN may, for example, be a convolutional PatchGAN classifier. The U-NET may comprise successive layers, where pooling operations may be replaced by upsampling operators. These layers may increase the resolution of the output of the U-NET.
According to one embodiment, the system is configured to connect to one or more MRI systems and to receive the input B1 map and/or the measured B1 maps from the MRI systems.
According to one embodiment, the system further comprises a MRI system, the MRI system being configured for acquiring image data and to reconstruct B1 maps out of the image data, the input B1 map and/or the measured B1 maps comprise the reconstructed B1 maps.
In another aspect, the invention relates to a method for determining electrical properties, EP, of a target volume in a subject. The method comprises: performing a first training of a deep neural network, DNN, using a first training dataset, the first training dataset comprising training B1 field maps and corresponding first EP maps, the first training resulting in a pre-trained DNN configured for generating EP maps from B1 field maps; performing a second training of the pre-trained DNN using conditional generative adversarial networks, GAN, and a second training dataset, wherein the pre-trained DNN is a generator of the conditional GAN, the second training dataset comprising measured B1 maps and second EP maps, the second training resulting in a trained DNN; receiving an input B1 field map of the target volume and generating an EP map of the input B1 field map using the trained DNN.
In another aspect, the invention relates to a computer program product comprising machine executable instructions for execution by a processor, wherein execution of the machine executable instructions causes the processor to the methods of any of the preceding embodiments.
It is understood that one or more of the aforementioned embodiments of the invention may be combined as long as the combined embodiments are not mutually exclusive.
In the following preferred embodiments of the invention will be described, by way of example only, and with reference to the drawings in which:
In the following, like numbered elements in the figures are either similar elements or perform an equivalent function. Elements which have been discussed previously will not necessarily be discussed in later figures if the function is equivalent.
Various structures, systems and devices are schematically depicted in the figures for purposes of explanation only and so as to not obscure the present invention with details that are well known to those skilled in the art. Nevertheless, the attached figures are included to describe and explain illustrative examples of the disclosed subject matter.
It will be appreciated that the methods described herein are at least partly non-interactive, and automated by way of computerized systems. For example, these methods can further be implemented in software 121, (including firmware), hardware, or a combination thereof. In exemplary embodiments, the methods described herein are implemented in software, as an executable program, and is executed by a special or general-purpose digital computer, such as a personal computer, workstation, minicomputer, or mainframe computer.
The processor 103 is a hardware device for executing software, particularly that stored in memory 107. The processor 103 can be any custom made or commercially available processor, a central processing unit (CPU), an auxiliary processor among several processors associated with the control system 111, a semiconductor based microprocessor (in the form of a microchip or chip set), a micro-processor, or generally any device for executing software instructions. The processor 103 may control the operation of the scanning imaging system 101.
The memory 107 can include any one or combination of volatile memory elements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM, etc.)) and non-volatile memory elements (e.g., ROM, erasable programmable read only memory (EPROM), electronically erasable programmable read only memory (EEPROM), programmable read only memory (PROM). Note that the memory 107 can have a distributed architecture, where various components are situated remote from one another, but can be accessed by the processor 103. Memory 107 may store an instruction or data related to at least one other constituent element of the medical analysis system 100.
The control system 111 may further comprise a display device 125 which displays characters and images and the like e.g. on a user interface 129. The display device 125 may be a touch screen display device.
The medical analysis system 100 may further comprise a power supply 108 for powering the medical analysis system 100. The power supply 108 may for example be a battery or an external source of power, such as electricity supplied by a standard AC outlet.
The scanning imaging system 101 may comprise at least one of MRI, CT and PET-CT imagers. The control system 111 and the scanning imaging system 101 may or may not be an integral part. In other terms, the control system 111 may or may not be external to the scanning imaging system 101.
The scanning imaging system 101 comprises components that may be controlled by the processor 103 in order to configure the scanning imaging system 101 to provide image data to the control system 111. The configuration of the scanning imaging system 101 may enable the operation of the scanning imaging system 101. The operation of the scanning imaging system 101 may for example be automatic.
The connection between the control system 111 and the scanning imaging system 101 may for example comprise a BUS Ethernet connection, WAN connection, or Internet connection etc.
In one example, the scanning imaging system 101 may be configured to provide output data such as images in response to a specified measurement. The control system 111 may be configured to receive data such as MR image data from the scanning imaging system 101. For example, the processor 103 may be adapted to receive information (automatically or upon request) from the scanning imaging system 101 in a compatible digital form so that such information may be displayed on the display device 125. Such information may include operating parameters, alert notifications, and other information related to the use, operation and function of the scanning imaging system 101.
The medical analysis system 100 may be configured to communicate via a network 130 with other scanning imaging systems 131 and/or databases 133. The network 130 comprises for example a wireless local area network (WLAN) connection, WAN (Wide Area Network) connection LAN (Local Area Network) connection or a combination thereof. The databases 133 may comprise information relates to patients, scanning imaging systems, anatomies, scan geometries, scan parameters, scans etc. The databases 133 may for example comprise an electronic medical record (EMR) database comprising patients' EMR, Radiology Information System database, medical image database, PACS, Hospital Information System database and/or other databases comparing data that can be used for planning a scan geometry. The databases 133 may for example comprise training datasets for the training performed by the present subject matter.
The memory 107 may further comprise an artificial intelligence (AI) component 150. The AI component 150 may or may not be part of software component 121. The AI component 150 may be configured for training a DNN in accordance with the present subject matter and to provide the trained DNN for further use. For example, if the control system 111 is not part of the scanning imaging system 101, the trained DNN may be provided to the scanning imaging system 101 such that it can be used at the scanning imaging system 101 for determining EP maps.
In step 201, a first training of a DNN may be performed using a first training dataset. The first training dataset comprises training B1 field maps and corresponding first EP maps. The first training results in a pre-trained DNN configured for generating EP maps from B1 field maps. The first training may be a supervised pre-training of the DNN. The DNN may for example comprise a predefined number of layers, wherein the training may result in optimized values of the weights of the DNN after the first training.
The first training may be performed using simulation data. The first training dataset comprises simulated B1 field maps and associated simulated EP maps (the simulated EP maps may be labels of the simulated B1 field maps). The simulated EP maps may be obtained by EP models and may thus provide realistic labels. Using simulation data may be advantageous as it may increase accuracy and variability of training data. The accuracy relates to the degree in which B1 map and EPs fulfil the physical relationship determined by the Helmholtz equation. For example, by inheriting a potential ground truth's corruption, prediction accuracy of the trained model reaches the average accuracy of the ground truth at maximum. The variability refers to geometrical variability. Experiments show that neural networks are able to approximate the desired underlying functionality to great accuracy for prediction data that is similar to the training data. A large variability within the training set has a beneficial effect of increasing the likelihood of similarity between prediction data and training data.
The simulated B1 field maps may be provided without artifacts. This may be advantageous as a simulation of artifacts may not be accurate or reliable simulation. Due to the huge variety of possible in-vivo artifacts, a simulation of these artifacts may be too cumbersome.
The first training may be useful as it initializes the weights of the DNN to be trained in step 203.
In step 203, a second training of the pre-trained DNN may be performed using a GAN framework. The second training may be performed using a second training dataset. The second training may be performed such that the pre-trained DNN is the generator of the GAN framework. The second training dataset comprises measured B1 maps and second EP maps (the second EP maps are not labels of the measured B1 maps)
The measured B1 field maps (in-vivo B1 data) may comprise artifacts. Thus, along with electromagnetic information, in-vivo B1 data comprises artifacts related to image acquisition (e.g. noise, patient motion, blood or cerebrospinal fluid pulsation, system imperfections). This may improve the authenticity of the second training dataset used by the present subject matter. Authenticity of the training set means that type and abundance of these artifacts are comparable in training and prediction data. Prediction data may be input data used in the application stage of the trained DNN.
The second EP maps may be simulated EP maps. The simulated EP maps may be real (or realistic) EP maps in the context of training a cGAN. During the second training, the discriminator looks at the simulated EP maps and generated EP maps separately. The generated EP maps are EP maps generated from the measured B1 field maps by the generator. The discriminator may distinguish whether an input EP map is real or generated. The second training may enable that the generator is conditioned with measured B1 field maps. This may enable that, instead of generating EP maps from noise only, measured B1 field maps may be combined with noise and may be used by the generator to generate the EP maps. The training of the generator and discriminator of the GAN may then be performed in an adversarial way as with training the GAN, but the generator is conditioned with the measured B1 field maps.
The second training results in a trained DNN.
The first and second training sets may be built using existing B1 field maps of a volume. That volume may be the target volume or another subject's volume.
In step 205, an input B1 field map of the target volume may be received and an EP map of the input B1 field map may be generated using the trained DNN.
The present method uses an adversarial domain adaptation, for attaining robustness of networks trained on simulations towards realistic in-vivo artifacts. As described above, in addition to the reconstructing network (generator), a second neural network (discriminator) is utilized. After supervised pretraining of the generator with simulations (without in-vivo artifacts) and respective labels, training is continued in an unsupervised way. Subsequent model improvements are guided by the discriminator by using an adversarial training strategy. While the discriminator aims at distinguishing between reconstruction output of the generator and realistic images, the generator is trained to fool the discriminator into evaluating its output as realistic. Discriminator and generator are trained alternately with model parameters of the respective other network frozen. Thereby, improvements in one network allow for more precise updates of the other and vice versa. After training, only the generator is used for prediction, as the discriminator is exclusively trained for enhancing the generator.
In the pre-training stage 301, the DNN 310 may be trained using a first training set. The first training set comprises simulated B1 field maps 320 and corresponding simulated electrical properties 321. The simulated B1 field maps 320 may be provided without artifacts. The generator 310 is pre-trained with simulated B1 data and EP labels related to the simulations model. This results in the pre-trained DNN 312.
In the adversarial training stage 303, the pre-trained DNN 312 is trained in a conditional GAN framework using a second training set. The cGAN framework comprises a discriminator 313. The discriminator 313 may be a DNN. The second training set comprises measured B1 field maps 323. The measured B1 field maps comprise artifacts. The second training set further comprises simulated electrical properties 326. The generator 312 is configured to generate from the measured B1 field maps 323 reconstructed electrical properties 325 during the training. The training of the generator 312 is performed in an adversarial way with the discriminator 313. The discriminator 313 is trained to distinguish between realistic EPs 326 and the generator output 325 from artifact encumbered data. The generator 312 is trained in alternation with the discriminator 313 with the opposite goal of fooling the discriminator by creating realistic looking EP reconstructions. The adversarial training stage 303 results in a trained DNN 314. Thus, after adversarial training the robust generator 314 can be applied to measured B1 data.
In the application stage 305, the trained DNN 314 may be used to predict or reconstruct electrical properties 328 of input measured B1 field maps 327.
This process of
Within the bore 706 of the magnet there is also a set of magnetic field gradient coils 710 which is used during acquisition of magnetic resonance data to spatially encode magnetic spins of a target volume within the imaging volume or examination volume 708 of the magnet 704. The magnetic field gradient coils 710 are connected to a magnetic field gradient coil power supply 712. The magnetic field gradient coils 710 are intended to be representative. Typically, magnetic field gradient coils 710 contain three separate sets of coils for the encoding in three orthogonal spatial directions. A magnetic field gradient power supply supplies current to the magnetic field gradient coils. The current supplied to the magnetic field gradient coils 710 is controlled as a function of time and may be ramped or pulsed.
MRI system 700 further comprises an RF coil 714 at the subject 718 and adjacent to the examination volume 708 for generating RF excitation pulses. The RF coil 714 may include for example a set of surface coils or other specialized RF coils. The RF coil 714 may be used alternately for transmission of RF pulses as well as for reception of magnetic resonance signals e.g., the RF coil 714 may be implemented as a transmit array coil comprising a plurality of RF transmit coils. The RF coil 714 is connected to one or more RF amplifiers 715.
The magnetic field gradient coil power supply 712 and the RF amplifier 715 are connected to a hardware interface of control system 111. The memory 107 of control system 111 may for example comprise a control module. The control module contains computer-executable code which enables the processor 103 to control the operation and function of the magnetic resonance imaging system 700. It also enables the basic operations of the magnetic resonance imaging system 700 such as the acquisition of magnetic resonance data.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as an apparatus, method or computer program product. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a ‘circuit’, ‘module’ or ‘system’. Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer executable code embodied thereon.
Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A ‘computer-readable storage medium’ as used herein encompasses any tangible storage medium which may store instructions which are executable by a processor of a computing device. The computer-readable storage medium may be referred to as a computer-readable non-transitory storage medium. The computer-readable storage medium may also be referred to as a tangible computer readable medium. In some embodiments, a computer-readable storage medium may also be able to store data which is able to be accessed by the processor of the computing device. Examples of computer-readable storage media include, but are not limited to: a floppy disk, a magnetic hard disk drive, a solid state hard disk, flash memory, a USB thumb drive, Random Access Memory (RAM), Read Only Memory (ROM), an optical disk, a magneto-optical disk, and the register file of the processor. Examples of optical disks include Compact Disks (CD) and Digital Versatile Disks (DVD), for example CD-ROM, CD-RW, CD-R, DVD-ROM, DVD-RW, or DVD-R disks. The term computer readable-storage medium also refers to various types of recording media capable of being accessed by the computer device via a network or communication link. For example, a data may be retrieved over a modem, over the internet, or over a local area network. Computer executable code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with computer executable code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
A ‘computer memory’ or ‘memory’ is an example of a computer-readable storage medium. A computer memory is any memory which is directly accessible to a processor. A ‘computer storage’ or ‘storage’ is a further example of a computer-readable storage medium. A computer storage is any non-volatile computer-readable storage medium. In some embodiments computer storage may also be computer memory or vice versa.
A ‘processor’ as used herein encompasses an electronic component which is able to execute a program or machine executable instruction or computer executable code. References to the computing device comprising ‘a processor’ should be interpreted as possibly containing more than one processor or processing core. The processor may for instance be a multi-core processor. A processor may also refer to a collection of processors within a single computer system or distributed amongst multiple computer systems. The term computing device should also be interpreted to possibly refer to a collection or network of computing devices each comprising a processor or processors. The computer executable code may be executed by multiple processors that may be within the same computing device or which may even be distributed across multiple computing devices.
Computer executable code may comprise machine executable instructions or a program which causes a processor to perform an aspect of the present invention. Computer executable code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the ‘C’ programming language or similar programming languages and compiled into machine executable instructions. In some instances, the computer executable code may be in the form of a high-level language or in a pre-compiled form and be used in conjunction with an interpreter which generates the machine executable instructions on the fly.
The computer executable code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
Aspects of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block or a portion of the blocks of the flowchart, illustrations, and/or block diagrams, can be implemented by computer program instructions in form of computer executable code when applicable. It is further understood that, when not mutually exclusive, combinations of blocks in different flowcharts, illustrations, and/or block diagrams may be combined. These computer program instructions may be provided to a processor of a general-purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
A ‘user interface’ as used herein is an interface which allows a user or operator to interact with a computer or computer system. A ‘user interface’ may also be referred to as a ‘human interface device’. A user interface may provide information or data to the operator and/or receive information or data from the operator. A user interface may enable input from an operator to be received by the computer and may provide output to the user from the computer. In other words, the user interface may allow an operator to control or manipulate a computer and the interface may allow the computer indicate the effects of the operator's control or manipulation. The display of data or information on a display or a graphical user interface is an example of providing information to an operator. The receiving of data through a keyboard, mouse, trackball, touchpad, pointing stick, graphics tablet, joystick, gamepad, webcam, headset, gear sticks, steering wheel, pedals, wired glove, dance pad, remote control, and accelerometer are all examples of user interface components which enable the receiving of information or data from an operator.
A ‘hardware interface’ as used herein encompasses an interface which enables the processor of a computer system to interact with and/or control an external computing device and/or apparatus. A hardware interface may allow a processor to send control signals or instructions to an external computing device and/or apparatus. A hardware interface may also enable a processor to exchange data with an external computing device and/or apparatus. Examples of a hardware interface include, but are not limited to: a universal serial bus, IEEE 1394 port, parallel port, IEEE 1284 port, serial port, RS-232 port, IEEE-488 port, Bluetooth connection, Wireless local area network connection, TCP/IP connection, Ethernet connection, control voltage interface, MIDI interface, analog input interface, and digital input interface.
A ‘display’ or ‘display device’ as used herein encompasses an output device or a user interface adapted for displaying images or data. A display may output visual, audio, and or tactile data. Examples of a display include, but are not limited to: a computer monitor, a television screen, a touch screen, tactile electronic display, Braille screen,
Cathode ray tube (CRT), Storage tube, Bistable display, Electronic paper, Vector display, Flat panel display, Vacuum fluorescent display (VF), Light-emitting diode (LED) displays, Electroluminescent display (ELD), Plasma display panels (PDP), Liquid crystal display (LCD), Organic light-emitting diode displays (OLED), a projector, and Head-mounted display.
While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments.
Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word ‘comprising’ does not exclude other elements or steps, and the indefinite article ‘a’ or ‘an’ does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measured cannot be used to advantage. A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be construed as limiting the scope.
| Number | Date | Country | Kind |
|---|---|---|---|
| 19185123.7 | Jul 2019 | EP | regional |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2020/068435 | 7/1/2020 | WO |
