Patent Application
20200357524
The present invention relates to an apparatus for patient data availability analysis, to a system for patient data availability analysis, to a method for patient data availability analysis, as well as to a computer program element.
The general background of this invention is the field of identifying patient records. Cardiovascular Information Systems (CVIS) have been available for many years in the healthcare industry. One activity of utilizing information and preferably knowledge from CVIS or Clinical Data repositories (CDR) is to find historical cases that match certain new patient or query criteria so as to provide valuable reference to the decision maker, e.g. the care giver or the physician. There is the trend of data analytics done on collected data from CDR, which provide tremendous valuable data without the heavy dedicated cost and labor of formal clinical studies. For a typical clinical research problem, a number of variables (features) are needed to be extracted from a CDR and there can be multiple ways to obtain these variables from different sources through different rules, yet resulting in different data availability due to noises in the data. CDR data contains noise because clinical practice necessarily balances the treatment of patients, in a timely manner, with the making and recording of absolutely complete and accurate with respect to the patient and the treatment. In addition, due to humans being involved in the processing and recording of data, noise is again introduced due to such incomplete data.
US2016/0019265A1 describes techniques for generating automated advice with respect to consolidating a plurality of sources. In an embodiment, a set of one or more parameters relating to a proposed consolidation for a plurality of consolidation sources is received. In response to receiving the set of one or more parameters, a set of one or more recommendations for consolidating the plurality of consolidation sources is generated and stored on at least one of a volatile or non-volatile computer-readable storage medium. In some embodiments, the set of one or more recommendations may indicate how to improve a performance associated with consolidating the plurality of sources to a set of one or more destinations based on a particular consolidation scenario. The set of one or more recommendations may be displayed during consolidation planning for the plurality of consolidation sources.
WO 2014/201515 A1 discloses systems and processes for processing medical data, e.g., for determining a likelihood, or risk, of an adverse event or outcome for a person based on machine learning techniques. The outcome may be, for example, a risk of attempting suicide, a probability of cancer survival, a number of re-hospitalisations, etc.
It would be advantageous to have an improved apparatus and associated method for patient data availability analysis, and an improved system for patient data availability analysis, to enable self-service of clinical research data records that includes noisy practice data.
The object of the present invention is solved with the subject matter of the independent claims, wherein further embodiments are incorporated in the dependent claims. It should be noted that the following described aspects and examples of the invention apply also for the apparatus for patient data availability analysis, the system for patient data availability analysis, the method for patient data availability analysis, and for the computer program element.
According to a first aspect, there is provided an apparatus for patient data availability analysis, comprising:
The input module is configured to enable a user to specify a plurality of models. Each model provides output as a function of model variables. The input module is configured also to enable the user to define at least some model variables for the plurality of models. The input module is configured also to enable the user to specify source variables. A model variable can be derived from one or more source variables. The input module is configured also to enable the user to specify at least one data source. The input module is configured also to receive a plurality of data records from the at least one data source. Each data record comprises at least one attribute. The data and model analysis module is configured to determine a set of available data records for each model from the plurality of data records. The determination for a model comprises utilization of the model variables for that model, the associated source variables and the at least one attribute for the plurality of data records. The data and model analysis module is configured also to determine a plurality of availability measures for the corresponding plurality of models, the determination comprising utilization of the determined set of available data records for each model. The optimization module is configured to rank and select a sub-set of models of the plurality of models as top models of data availability. The selection comprises utilization of the plurality of availability measures.
In this manner, users can conveniently understand the trade-off between desired input variables and the data size that supports the set of desired variables. They can obtain the optimized availability of data according to their requirements on data characteristics. They can also identify potential data quality issues by variable-availability profiling, and check if the data supports their study hypothesis to be further validated by follow-up data analytics.
To put this another way, data availability is analysed during shortlisting among multiple hypotheses (models), enabling further prediction and modelling steps using that available data and shortlisted models to be more efficient and effective.
Thus, data availability selection at the first level is carried out to determine the best availability among different variable source and derivation choices to simplify and optimize decisions on follow-up analysis, otherwise each model would have non-trivial combinations to be selected and arbitrary choices would introduce challenges of data quality in later modelling steps.
In an example. the input module is configured to enable a user to provide constraints of expected data, and the data and model analysis module is configured to determine for each model of the plurality of models statistics of variables specified in the constraints of expected data provided by the user. The selection of the sub-set of models can then comprise utilization of the determined statistics of variables for the plurality of models.
In an example, the optimization module is configured to output information on whether any data records of the available data records for the top models are outside of the constraints of expected data input by the user.
In other words, the apparatus outputs the information of compromised user expectations (violated constraints), enabling the user to better determine if a model and its associated available data is acceptable.
In an example, the data and model analysis module is configured to determine at least one model performance for at least one model of the plurality of models. The selection of the sub-set of models can then comprise utilization of the at least one model performance.
In an example, the optimization module is configured to determine a best model of the sub-set of models.
In an example, the optimization module is configured to rank the sub-set of models.
In an example, the data and model analysis module is configured to determine a number of clean data records of the available data records for each model, wherein each of the available data records comprises a plurality of values. The clean data records can then be determined as those data records of the available data records that are missing less than a threshold number of values, and the availability measure for a model can comprise the number of clean data records for that model.
In an example, the availability measure for the model comprises the number of clean data records for that model divided by the total number of clean data records.
In an example, the threshold value is a percentage of the number of values, which comprises zero percent.
Thus, clean data records can be those data records that have no missing values, or those data records that have a number of missing values that is below a threshold percentage.
In an example, the availability measure for a model has a value that is weighted by the number of model variables for that model.
In an example, the data and model analysis module is configured to derive at least one model variable for at least one of the plurality of models. The derivation comprises matching at least some of the model variables input by the user with corresponding at least one source variable.
In this way, the apparatus can determine the correct model variables to use for a model on the basis of the information provided by the user.
In an example, the input module is configured to enable the user to input a search query. The user specifying the plurality of models can comprise the input unit identifying at least one model according to the search query.
In other words, the user can explicitly define the models to be used, but can also provide search queries and the input unit retrieves existing models, such as risk models, according to the user search or key words, from knowledge bases in the clinical domain, and thus simplifies the input of model variable and variable derivation rules.
According to a second aspect, there is provided a system for patient data availability analysis, comprising:
The plurality patient records are provided from the at least one data source to the input unit and the output unit is configured to output information relating to the top models of data availability.
According to a third aspect, there is provided a method for patient data availability analysis, comprising:
a) specifying a plurality of models, wherein each model provides output as a function of model variables;
b) defining at least some model variables for the plurality of models;
c) specifying source variables, wherein a model variable can be derived from one or more source variables;
d) specifying at least one data source;
e) receiving a plurality of data records from the at least one data source, wherein each data record comprises at least one attribute;
f) determining a set of available data records for each model from the plurality of data records, the determination for a model comprising utilizing the model variables for that model, the associated source variables and the at least one attribute for the plurality of data records;
g) determining a plurality of availability measures for the corresponding plurality of models, the determination comprising utilizing the determined set of available data records for each model; and
h) selecting a sub-set of models of the plurality of models as top models of data availability, wherein the selection comprises utilizing the plurality of availability measures.
Thus an advanced data analytics method can be a directly integrated module in a Cardiovascular Information Systems (CVIS)/Clinical Data Repository (CDR) to allow users/physician to turn the information stored into knowledge effective and efficiently.
According to another aspect, there is provided a computer program element controlling apparatus and/or system as previously described which, if the computer program element is executed by a processing unit, is adapted to perform the method steps as previously described.
There is also provided a computer readable medium having stored the computer element as previously described.
Advantageously, the benefits provided by any of the above aspects equally apply to all of the other aspects and vice versa.
The above aspects and examples will become apparent from and be elucidated with reference to the embodiments described hereinafter.
Exemplary embodiments will be described in the following with reference to the following drawings:
In an example, the at least one attribute value comprises one or more of: demographic information, life style information, medical information, care provider information, history and risk factor information, previous visit information, procedure information, etc. of a specific patient. The medical information includes a patient's basic health information, lesion information, device information and follow-up information. The at least one attribute can therefore can include demographic data (age, gender, weight, ethnicity, et cetera); presence/absence of chronic behavioural conditions (smoking, heavy alcohol consumption, consumption of various recreational drugs, et cetera); presence/absence of various chronic clinical conditions (high blood pressure, diabetes, asthma, heart disease; et cetera); presence/absence of various acute ailments (pneumonia or other acute respiratory ailments, various oncological conditions, or so forth); features related to same (e.g., cancer stage and grade); medical and care provider information and so forth. Thus, the at least one attribute value are data fields in a medical record, which can represent medical issues, for example, a symptom (e.g. bleeding), a severe medical result (e.g. death) or a medical status (e.g. normal), or other contextual information, for example, medical histories, lab test results or demographic data of a patient. The attribute value may be the value of contextual information or the likelihood of a medical issue for example.
In an example, the at least one data source comprises one or more clinical data repositories (CDR), lab information system (LIS), and admission notes (AN).
In this manner, the noise inherited including incompleteness and multiple potentially conflicting sources of data can be mitigated providing an efficient and effective means to help users understand and make optimized decisions on what hypotheses (models) to validate considering the status (availability) of CDR data for example.
According to an example, the input module is configured to enable a user to provide constraints of expected data. The data and model analysis module is configured to determine for each model of the plurality of models statistics of variables specified in the constraints of expected data provided by the user. The selection of the sub-set of models then comprises utilization of the determined statistics of variables for the plurality of models.
In an example, the statistics of variables specified in the user constraints comprises for example record count, gender, age in expected record number, gender ratios, age ranges.
According to an example, the optimization module is configured to output information on whether any data records of the available data records for the top models are outside of the constraints of expected data input by the user.
According to an example, the data and model analysis module is configured to determine at least one model performance for at least one model of the plurality of models. The selection of the sub-set of models then comprises utilization of the at least one model performance.
According to an example, the optimization module is configured to determine a best model of the sub-set of models.
According to an example, the optimization module is configured to rank the sub-set of models.
According to an example, the data and model analysis module is configured to determine a number of clean data records of the available data records for each model. Each of the available data records can comprise a plurality of values, and the clean data records can be determined as those data records of the available data records that are missing less than a threshold number of values. The availability measure for a model then comprises the number of clean data records for that model.
According to an example, the availability measure for the model comprises the number of clean data records for that model divided by the total number of clean data records.
According to an example, the threshold value is a percentage of the number of values, which comprises zero percent.
Thus for example, clean data records can be those data records that have no missing values, have less than 1% of missing values, have less than 2% of missing values, have less than 5% of missing values, have less than 10% of missing values, have less than 20% of missing values.
According to an example, the availability measure for a model has a value that is weighted by the number of model variables for that model.
In an example, the at least one data source comprises CDR, LIS and admission noted (AN).
According to an example, the data and model analysis module is configured to derive at least one model variable for at least one of the plurality of models. The derivation comprises matching at least some of the model variables input by the user with corresponding at least one source variable.
According to an example, the input module is configured to enable the user to input a search query. The user specifying the plurality of models can then comprise the input unit identifying at least one model according to the search query.
in a specifying step 210, also referred to as step a), specifying a plurality of models, wherein each model provides output as a function of model variables;
in a defining step 220, also referred to as step b), defining at least some model variables for the plurality of models;
in a specifying step 230, also referred to as step c), specifying source variables, wherein a model variable can be derived from one or more source variables;
in a specifying step 240, also referred to as step d), specifying at least one data source;
in a receiving step 250, also referred to as step e), receiving a plurality of data records from the at least one data source, wherein each data record comprises at least one attribute;
in a determining step 260, also referred to as step f), determining a set of available data records for each model from the plurality of data records, the determination for a model comprising utilizing the model variables for that model, the associated source variables and the at least one attribute for the plurality of data records;
in a determining step 270, also referred to as step g), determining a plurality of availability measures for the corresponding plurality of models, the determination comprising utilizing the determined set of available data records for each model; and in a selecting step 280, also referred to as step h), selecting a sub-set of models of the plurality of models as top models of data availability, wherein the selection comprises utilizing the plurality of availability measures.
In an example, wherein the method comprises providing constraints of expected data, and comprises determining for the top models statistics of variables specified in the constraints of expected data. The selection of the sub-set of models then comprises utilization of the determined statistics of variables.
In an example, the method comprises outputting information on whether any data records of the available data records for a model are outside of (do not satisfy) the constraints of expected data.
In an example, the method comprises determining at least one model performance for at least one model of the plurality of models. The selecting the sub-set of models then comprises utilizing the at least one model performance.
In an example, the method comprises determining a best model of the sub-set of models.
In an example, the method comprises ranking the sub-set of models.
In an example, the method comprises determining a number of clean data records of the available data records for each model, where each of the available data records comprises a plurality of values. The clean data records are then determined as those data records of the available data records that are missing less than a threshold number of values. The availability measure for a model then comprises the number of clean data records for that model.
In an example, determining the availability measure for the model comprises calculating the number of clean data records for that model divided by the total number of clean data records.
In an example of the method, the threshold value is a percentage of the number of values, which comprises zero percent.
In an example of the method, the availability measure for a model has a value that is weighted by the number of model variables for that model.
In an example, the method comprises deriving at least one model variable for at least one of the plurality of models, the derivation comprising matching at least some of the model variables with corresponding at least one source variable.
In an example, the method comprises inputting a search query, and wherein specifying the plurality of models comprise identifying at least one model according to the search query.
The apparatus, system and method for patient data availability analysis are now described in more detail in conjunction with
The apparatus, method and system for patient record identification as described with respect to
Typically a (clinical) research topic centers on validating a hypothesis (here also called a model), or shortlisting from relevant hypotheses, from data collected (or to be collected). Therefore, data availability is critical for whether the hypotheses can be validated before real prediction analysis and risk modeling can be done. In traditional designed studies, data was collected in a very strict manner, with dedicated efforts on quality, stringent inclusion/exclusion criteria and human intensive follow-ups. This way of working restricts the scalability and requires huge cost and labor for population-wide representative conclusions. On the other hand, with the widely adopted electronic records and health information systems, as well as in-depth CDR, hypotheses can be generated and validated using the tremendous existing data collected day-after-day. The challenge lies on the noise inherited including incompleteness and multiple potentially conflicting sources of data. The system of apparatus, system and method embodied in the detailed examples shown in
There are a number of challenges, at multiple levels, that are addressed:
Continuing with
To help understand the issues being addressed by the described apparatus, system and method for patient data availability analysis consider the following: Anemia (Yes/No) may be available directly from CDR with considerable missing values, and may also be derived by checking LIS if baseline hemoglobin <10 g/dL (100 g/L). However, different combinations of ways to derive single variables, multiplied by the various variables that require derivations, result in a non-trivial number of data availability combinations that users cannot possibly directly handle. For simplicity, it is assumed the models all have the same variable requirements. Imagine there are 5 models, each model has 8 variables, 2 out of 8 variables have each 4 total different ways of derivations, and the other 3 have each 2 possible sources. The remaining 3 variables are from the same source with no need of derivation. There are therefore 5×(42×23×13)=640 combinations. Adding each additional variable to build a new model from these choices leads to another level of multiplication (*2 each time if a new variable has two alternative ways). Furthermore, for a certain clinical relevant research study, the user (a clinician) would expect some requirements to be met, such as minimal patient numbers, gender ratios, age ranges, etc. This further complicates the feasibility and makes it unclear for the user to be confident to reach his study objectives: if hypotheses can be rigorously validated, or it fails to reach any conclusions simply due to unavailability of data.
The system of apparatus, system and method embodied in the detailed examples shown in
In another exemplary embodiment, a computer program or computer program element is provided that is characterized by being configured to execute the method steps of the method according to one of the preceding embodiments, on an appropriate system.
The computer program element might therefore be stored on a computer unit, which might also be part of an embodiment. This computing unit may be configured to perform or induce performing of the steps of the method described above. Moreover, it may be configured to operate the components of the above described apparatus and/or system. The computing unit can be configured to operate automatically and/or to execute the orders of a user. A computer program may be loaded into a working memory of a data processor. The data processor may thus be equipped to carry out the method according to one of the preceding embodiments.
This exemplary embodiment of the invention covers both, a computer program that right from the beginning uses the invention and computer program that by means of an update turns an existing program into a program that uses invention.
Further on, the computer program element might be able to provide all necessary steps to fulfill the procedure of an exemplary embodiment of the method as described above.
According to a further exemplary embodiment of the present invention, a computer readable medium, such as a CD-ROM, USB stick or the like, is presented wherein the computer readable medium has a computer program element stored on it which computer program element is described by the preceding section.
A computer program may be stored and/or distributed on a suitable medium, such as an optical storage medium or a solid state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems.
However, the computer program may also be presented over a network like the World Wide Web and can be downloaded into the working memory of a data processor from such a network. According to a further exemplary embodiment of the present invention, a medium for making a computer program element available for downloading is provided, which computer program element is arranged to perform a method according to one of the previously described embodiments of the invention.
It has to be noted that embodiments of the invention are described with reference to different subject matters. In particular, some embodiments are described with reference to method type claims whereas other embodiments are described with reference to the device type claims. However, a person skilled in the art will gather from the above and the following description that, unless otherwise notified, in addition to any combination of features belonging to one type of subject matter also any combination between features relating to different subject matters is considered to be disclosed with this application. However, all features can be combined providing synergetic effects that are more than the simple summation of the features.
While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing a claimed invention, from a study of the drawings, the disclosure, and the dependent claims.
In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single processor or other unit may fulfill the functions of several items re-cited in the claims. The mere fact that certain measures are re-cited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope.
| Number | Date | Country | Kind |
|---|---|---|---|
| PCT/CN2017/114205 | Dec 2017 | CN | national |
| 17210579.3 | Dec 2017 | EP | regional |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2018/081888 | 11/20/2018 | WO | 00 |
